diff --git a/README.md b/README.md index 46db7f1..93064a0 100644 --- a/README.md +++ b/README.md @@ -40,38 +40,37 @@ Password: openarchiver_demo ## ✨ Key Features -- **Universal Ingestion**: Connect to any email provider to perform initial bulk imports and maintain continuous, real-time synchronization. Ingestion sources include: +- **Universal Ingestion**: Connect to any email provider to perform initial bulk imports and maintain continuous, real-time synchronization. Ingestion sources include: + - IMAP connection + - Google Workspace + - Microsoft 365 + - PST files + - Zipped .eml files - - IMAP connection - - Google Workspace - - Microsoft 365 - - PST files - - Zipped .eml files - -- **Secure & Efficient Storage**: Emails are stored in the standard `.eml` format. The system uses deduplication and compression to minimize storage costs. All data is encrypted at rest. -- **Pluggable Storage Backends**: Support both local filesystem storage and S3-compatible object storage (like AWS S3 or MinIO). -- **Powerful Search & eDiscovery**: A high-performance search engine indexes the full text of emails and attachments (PDF, DOCX, etc.). -- **Thread discovery**: The ability to discover if an email belongs to a thread/conversation and present the context. -- **Compliance & Retention**: Define granular retention policies to automatically manage the lifecycle of your data. Place legal holds on communications to prevent deletion during litigation (TBD). -- **Comprehensive Auditing**: An immutable audit trail logs all system activities, ensuring you have a clear record of who accessed what and when (TBD). +- **Secure & Efficient Storage**: Emails are stored in the standard `.eml` format. The system uses deduplication and compression to minimize storage costs. All data is encrypted at rest. +- **Pluggable Storage Backends**: Support both local filesystem storage and S3-compatible object storage (like AWS S3 or MinIO). +- **Powerful Search & eDiscovery**: A high-performance search engine indexes the full text of emails and attachments (PDF, DOCX, etc.). +- **Thread discovery**: The ability to discover if an email belongs to a thread/conversation and present the context. +- **Compliance & Retention**: Define granular retention policies to automatically manage the lifecycle of your data. Place legal holds on communications to prevent deletion during litigation (TBD). +- **Comprehensive Auditing**: An immutable audit trail logs all system activities, ensuring you have a clear record of who accessed what and when (TBD). ## 🛠️ Tech Stack Open Archiver is built on a modern, scalable, and maintainable technology stack: -- **Frontend**: SvelteKit with Svelte 5 -- **Backend**: Node.js with Express.js & TypeScript -- **Job Queue**: BullMQ on Redis for robust, asynchronous processing. (We use Valkey as the Redis service in the Docker Compose deployment mode, but you can use Redis as well.) -- **Search Engine**: Meilisearch for blazingly fast and resource-efficient search -- **Database**: PostgreSQL for metadata, user management, and audit logs -- **Deployment**: Docker Compose deployment +- **Frontend**: SvelteKit with Svelte 5 +- **Backend**: Node.js with Express.js & TypeScript +- **Job Queue**: BullMQ on Redis for robust, asynchronous processing. (We use Valkey as the Redis service in the Docker Compose deployment mode, but you can use Redis as well.) +- **Search Engine**: Meilisearch for blazingly fast and resource-efficient search +- **Database**: PostgreSQL for metadata, user management, and audit logs +- **Deployment**: Docker Compose deployment ## 📦 Deployment ### Prerequisites -- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) -- A server or local machine with at least 4GB of RAM (2GB of RAM if you use external Postgres, Redis (Valkey) and Meilisearch instances). +- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) +- A server or local machine with at least 4GB of RAM (2GB of RAM if you use external Postgres, Redis (Valkey) and Meilisearch instances). ### Installation @@ -106,17 +105,17 @@ Open Archiver is built on a modern, scalable, and maintainable technology stack: After deploying the application, you will need to configure one or more ingestion sources to begin archiving emails. Follow our detailed guides to connect to your email provider: -- [Connecting to Google Workspace](https://docs.openarchiver.com/user-guides/email-providers/google-workspace.html) -- [Connecting to Microsoft 365](https://docs.openarchiver.com/user-guides/email-providers/imap.html) -- [Connecting to a Generic IMAP Server](https://docs.openarchiver.com/user-guides/email-providers/imap.html) +- [Connecting to Google Workspace](https://docs.openarchiver.com/user-guides/email-providers/google-workspace.html) +- [Connecting to Microsoft 365](https://docs.openarchiver.com/user-guides/email-providers/imap.html) +- [Connecting to a Generic IMAP Server](https://docs.openarchiver.com/user-guides/email-providers/imap.html) ## 🤝 Contributing We welcome contributions from the community! -- **Reporting Bugs**: If you find a bug, please open an issue on our GitHub repository. -- **Suggesting Enhancements**: Have an idea for a new feature? We'd love to hear it. Open an issue to start the discussion. -- **Code Contributions**: If you'd like to contribute code, please fork the repository and submit a pull request. +- **Reporting Bugs**: If you find a bug, please open an issue on our GitHub repository. +- **Suggesting Enhancements**: Have an idea for a new feature? We'd love to hear it. Open an issue to start the discussion. +- **Code Contributions**: If you'd like to contribute code, please fork the repository and submit a pull request. Please read our `CONTRIBUTING.md` file for more details on our code of conduct and the process for submitting pull requests. diff --git a/docker-compose.yml b/docker-compose.yml index 64831eb..f38eea2 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,74 +1,74 @@ version: '3.8' services: - open-archiver: - image: logiclabshq/open-archiver:latest - container_name: open-archiver - restart: unless-stopped - ports: - - '4000:4000' # Backend - - '3000:3000' # Frontend - env_file: - - .env - volumes: - - archiver-data:/var/data/open-archiver - depends_on: - - postgres - - valkey - - meilisearch - networks: - - open-archiver-net + open-archiver: + image: logiclabshq/open-archiver:latest + container_name: open-archiver + restart: unless-stopped + ports: + - '4000:4000' # Backend + - '3000:3000' # Frontend + env_file: + - .env + volumes: + - archiver-data:/var/data/open-archiver + depends_on: + - postgres + - valkey + - meilisearch + networks: + - open-archiver-net - postgres: - image: postgres:17-alpine - container_name: postgres - restart: unless-stopped - environment: - POSTGRES_DB: ${POSTGRES_DB:-open_archive} - POSTGRES_USER: ${POSTGRES_USER:-admin} - POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-password} - volumes: - - pgdata:/var/lib/postgresql/data - ports: - - '5432:5432' - networks: - - open-archiver-net + postgres: + image: postgres:17-alpine + container_name: postgres + restart: unless-stopped + environment: + POSTGRES_DB: ${POSTGRES_DB:-open_archive} + POSTGRES_USER: ${POSTGRES_USER:-admin} + POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-password} + volumes: + - pgdata:/var/lib/postgresql/data + ports: + - '5432:5432' + networks: + - open-archiver-net - valkey: - image: valkey/valkey:8-alpine - container_name: valkey - restart: unless-stopped - command: valkey-server --requirepass ${REDIS_PASSWORD} - ports: - - '6379:6379' - volumes: - - valkeydata:/data - networks: - - open-archiver-net + valkey: + image: valkey/valkey:8-alpine + container_name: valkey + restart: unless-stopped + command: valkey-server --requirepass ${REDIS_PASSWORD} + ports: + - '6379:6379' + volumes: + - valkeydata:/data + networks: + - open-archiver-net - meilisearch: - image: getmeili/meilisearch:v1.15 - container_name: meilisearch - restart: unless-stopped - environment: - MEILI_MASTER_KEY: ${MEILI_MASTER_KEY:-aSampleMasterKey} - ports: - - '7700:7700' - volumes: - - meilidata:/meili_data - networks: - - open-archiver-net + meilisearch: + image: getmeili/meilisearch:v1.15 + container_name: meilisearch + restart: unless-stopped + environment: + MEILI_MASTER_KEY: ${MEILI_MASTER_KEY:-aSampleMasterKey} + ports: + - '7700:7700' + volumes: + - meilidata:/meili_data + networks: + - open-archiver-net volumes: - pgdata: - driver: local - valkeydata: - driver: local - meilidata: - driver: local - archiver-data: - driver: local + pgdata: + driver: local + valkeydata: + driver: local + meilidata: + driver: local + archiver-data: + driver: local networks: - open-archiver-net: - driver: bridge + open-archiver-net: + driver: bridge diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts index c0ddfb3..0a7d459 100644 --- a/docs/.vitepress/config.mts +++ b/docs/.vitepress/config.mts @@ -1,71 +1,80 @@ import { defineConfig } from 'vitepress'; export default defineConfig({ - head: [ - [ - 'script', - { - defer: '', - src: 'https://analytics.zenceipt.com/script.js', - 'data-website-id': '2c8b452e-eab5-4f82-8ead-902d8f8b976f' - } - ] - ], - title: 'Open Archiver', - description: 'Official documentation for the Open Archiver project.', - themeConfig: { - search: { - provider: 'local' - }, - logo: { - src: '/logo-sq.svg' - }, - nav: [ - { text: 'Home', link: '/' }, - { text: 'Github', link: 'https://github.com/LogicLabs-OU/OpenArchiver' }, - { text: "Website", link: 'https://openarchiver.com/' }, - { text: "Discord", link: 'https://discord.gg/MTtD7BhuTQ' } - ], - sidebar: [ - { - text: 'User Guides', - items: [ - { text: 'Get Started', link: '/' }, - { text: 'Installation', link: '/user-guides/installation' }, - { - text: 'Email Providers', - link: '/user-guides/email-providers/', - collapsed: true, - items: [ - { text: 'Generic IMAP Server', link: '/user-guides/email-providers/imap' }, - { text: 'Google Workspace', link: '/user-guides/email-providers/google-workspace' }, - { text: 'Microsoft 365', link: '/user-guides/email-providers/microsoft-365' }, - { text: 'EML Import', link: '/user-guides/email-providers/eml' }, - { text: 'PST Import', link: '/user-guides/email-providers/pst' } - ] - } - ] - }, - { - text: 'API Reference', - items: [ - { text: 'Overview', link: '/api/' }, - { text: 'Authentication', link: '/api/authentication' }, - { text: 'Auth', link: '/api/auth' }, - { text: 'Archived Email', link: '/api/archived-email' }, - { text: 'Dashboard', link: '/api/dashboard' }, - { text: 'Ingestion', link: '/api/ingestion' }, - { text: 'Search', link: '/api/search' }, - { text: 'Storage', link: '/api/storage' } - ] - }, - { - text: 'Services', - items: [ - { text: 'Overview', link: '/services/' }, - { text: 'Storage Service', link: '/services/storage-service' } - ] - } - ] - } + head: [ + [ + 'script', + { + defer: '', + src: 'https://analytics.zenceipt.com/script.js', + 'data-website-id': '2c8b452e-eab5-4f82-8ead-902d8f8b976f', + }, + ], + ], + title: 'Open Archiver', + description: 'Official documentation for the Open Archiver project.', + themeConfig: { + search: { + provider: 'local', + }, + logo: { + src: '/logo-sq.svg', + }, + nav: [ + { text: 'Home', link: '/' }, + { text: 'Github', link: 'https://github.com/LogicLabs-OU/OpenArchiver' }, + { text: 'Website', link: 'https://openarchiver.com/' }, + { text: 'Discord', link: 'https://discord.gg/MTtD7BhuTQ' }, + ], + sidebar: [ + { + text: 'User Guides', + items: [ + { text: 'Get Started', link: '/' }, + { text: 'Installation', link: '/user-guides/installation' }, + { + text: 'Email Providers', + link: '/user-guides/email-providers/', + collapsed: true, + items: [ + { + text: 'Generic IMAP Server', + link: '/user-guides/email-providers/imap', + }, + { + text: 'Google Workspace', + link: '/user-guides/email-providers/google-workspace', + }, + { + text: 'Microsoft 365', + link: '/user-guides/email-providers/microsoft-365', + }, + { text: 'EML Import', link: '/user-guides/email-providers/eml' }, + { text: 'PST Import', link: '/user-guides/email-providers/pst' }, + ], + }, + ], + }, + { + text: 'API Reference', + items: [ + { text: 'Overview', link: '/api/' }, + { text: 'Authentication', link: '/api/authentication' }, + { text: 'Auth', link: '/api/auth' }, + { text: 'Archived Email', link: '/api/archived-email' }, + { text: 'Dashboard', link: '/api/dashboard' }, + { text: 'Ingestion', link: '/api/ingestion' }, + { text: 'Search', link: '/api/search' }, + { text: 'Storage', link: '/api/storage' }, + ], + }, + { + text: 'Services', + items: [ + { text: 'Overview', link: '/services/' }, + { text: 'Storage Service', link: '/services/storage-service' }, + ], + }, + ], + }, }); diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 71c69d8..71b15f3 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -2,16 +2,16 @@ ## User guides -- [Get started](index.md) -- [Installation](user-guides/installation.md) -- [email-providers](user-guides/email-providers/index.md) - - [Connecting to Google Workspace](user-guides/email-providers/google-workspace.md) - - [Connecting to a Generic IMAP Server](user-guides/email-providers/imap.md) - - [Connecting to Microsoft 365](user-guides/email-providers/microsoft-365.md) +- [Get started](index.md) +- [Installation](user-guides/installation.md) +- [email-providers](user-guides/email-providers/index.md) + - [Connecting to Google Workspace](user-guides/email-providers/google-workspace.md) + - [Connecting to a Generic IMAP Server](user-guides/email-providers/imap.md) + - [Connecting to Microsoft 365](user-guides/email-providers/microsoft-365.md) --- -- [api](api/index.md) - - [Ingestion Sources API Documentation](api/ingestion.md) -- [services](services/index.md) - - [Pluggable Storage Service (StorageService)](services/storage-service.md) +- [api](api/index.md) + - [Ingestion Sources API Documentation](api/ingestion.md) +- [services](services/index.md) + - [Pluggable Storage Service (StorageService)](services/storage-service.md) diff --git a/docs/api/archived-email.md b/docs/api/archived-email.md index 625581f..601625c 100644 --- a/docs/api/archived-email.md +++ b/docs/api/archived-email.md @@ -27,29 +27,27 @@ Retrieves a paginated list of archived emails for a specific ingestion source. #### Responses -- **200 OK:** A paginated list of archived emails. +- **200 OK:** A paginated list of archived emails. ```json { - "items": [ - { - "id": "email-id", - "subject": "Test Email", - "from": "sender@example.com", - "sentAt": "2023-10-27T10:00:00.000Z", - "hasAttachments": true, - "recipients": [ - { "name": "Recipient 1", "email": "recipient1@example.com" } - ] - } - ], - "total": 100, - "page": 1, - "limit": 10 + "items": [ + { + "id": "email-id", + "subject": "Test Email", + "from": "sender@example.com", + "sentAt": "2023-10-27T10:00:00.000Z", + "hasAttachments": true, + "recipients": [{ "name": "Recipient 1", "email": "recipient1@example.com" }] + } + ], + "total": 100, + "page": 1, + "limit": 10 } ``` -- **500 Internal Server Error:** An unexpected error occurred. +- **500 Internal Server Error:** An unexpected error occurred. ### GET /api/v1/archived-emails/:id @@ -65,32 +63,30 @@ Retrieves a single archived email by its ID, including its raw content and attac #### Responses -- **200 OK:** The archived email details. +- **200 OK:** The archived email details. ```json { - "id": "email-id", - "subject": "Test Email", - "from": "sender@example.com", - "sentAt": "2023-10-27T10:00:00.000Z", - "hasAttachments": true, - "recipients": [ - { "name": "Recipient 1", "email": "recipient1@example.com" } - ], - "raw": "...", - "attachments": [ - { - "id": "attachment-id", - "filename": "document.pdf", - "mimeType": "application/pdf", - "sizeBytes": 12345 - } - ] + "id": "email-id", + "subject": "Test Email", + "from": "sender@example.com", + "sentAt": "2023-10-27T10:00:00.000Z", + "hasAttachments": true, + "recipients": [{ "name": "Recipient 1", "email": "recipient1@example.com" }], + "raw": "...", + "attachments": [ + { + "id": "attachment-id", + "filename": "document.pdf", + "mimeType": "application/pdf", + "sizeBytes": 12345 + } + ] } ``` -- **404 Not Found:** The archived email with the specified ID was not found. -- **500 Internal Server Error:** An unexpected error occurred. +- **404 Not Found:** The archived email with the specified ID was not found. +- **500 Internal Server Error:** An unexpected error occurred. ## Service Methods @@ -98,14 +94,14 @@ Retrieves a single archived email by its ID, including its raw content and attac Retrieves a paginated list of archived emails from the database for a given ingestion source. -- **ingestionSourceId:** The ID of the ingestion source. -- **page:** The page number for pagination. -- **limit:** The number of items per page. -- **Returns:** A promise that resolves to a `PaginatedArchivedEmails` object. +- **ingestionSourceId:** The ID of the ingestion source. +- **page:** The page number for pagination. +- **limit:** The number of items per page. +- **Returns:** A promise that resolves to a `PaginatedArchivedEmails` object. ### `getArchivedEmailById(emailId: string): Promise` Retrieves a single archived email by its ID, including its raw content and attachments. -- **emailId:** The ID of the archived email. -- **Returns:** A promise that resolves to an `ArchivedEmail` object or `null` if not found. +- **emailId:** The ID of the archived email. +- **Returns:** A promise that resolves to an `ArchivedEmail` object or `null` if not found. diff --git a/docs/api/auth.md b/docs/api/auth.md index f211d4a..5a574c8 100644 --- a/docs/api/auth.md +++ b/docs/api/auth.md @@ -21,40 +21,40 @@ Authenticates a user and returns a JWT if the credentials are valid. #### Responses -- **200 OK:** Authentication successful. +- **200 OK:** Authentication successful. ```json { - "accessToken": "your.jwt.token", - "user": { - "id": "user-id", - "email": "user@example.com", - "role": "user" - } + "accessToken": "your.jwt.token", + "user": { + "id": "user-id", + "email": "user@example.com", + "role": "user" + } } ``` -- **400 Bad Request:** Email or password not provided. +- **400 Bad Request:** Email or password not provided. ```json { - "message": "Email and password are required" + "message": "Email and password are required" } ``` -- **401 Unauthorized:** Invalid credentials. +- **401 Unauthorized:** Invalid credentials. ```json { - "message": "Invalid credentials" + "message": "Invalid credentials" } ``` -- **500 Internal Server Error:** An unexpected error occurred. +- **500 Internal Server Error:** An unexpected error occurred. ```json { - "message": "An internal server error occurred" + "message": "An internal server error occurred" } ``` @@ -64,21 +64,21 @@ Authenticates a user and returns a JWT if the credentials are valid. Compares a plain-text password with a hashed password to verify its correctness. -- **password:** The plain-text password. -- **hash:** The hashed password to compare against. -- **Returns:** A promise that resolves to `true` if the password is valid, otherwise `false`. +- **password:** The plain-text password. +- **hash:** The hashed password to compare against. +- **Returns:** A promise that resolves to `true` if the password is valid, otherwise `false`. ### `login(email: string, password: string): Promise` Handles the user login process. It finds the user by email, verifies the password, and generates a JWT upon successful authentication. -- **email:** The user's email. -- **password:** The user's password. -- **Returns:** A promise that resolves to a `LoginResponse` object containing the `accessToken` and `user` details, or `null` if authentication fails. +- **email:** The user's email. +- **password:** The user's password. +- **Returns:** A promise that resolves to a `LoginResponse` object containing the `accessToken` and `user` details, or `null` if authentication fails. ### `verifyToken(token: string): Promise` Verifies the authenticity and expiration of a JWT. -- **token:** The JWT string to verify. -- **Returns:** A promise that resolves to the token's `AuthTokenPayload` if valid, otherwise `null`. +- **token:** The JWT string to verify. +- **Returns:** A promise that resolves to the token's `AuthTokenPayload` if valid, otherwise `null`. diff --git a/docs/api/authentication.md b/docs/api/authentication.md index b284a2c..6e6295a 100644 --- a/docs/api/authentication.md +++ b/docs/api/authentication.md @@ -22,12 +22,12 @@ Content-Type: application/json ```json { - "accessToken": "your.jwt.token", - "user": { - "id": "user-id", - "email": "user@example.com", - "role": "user" - } + "accessToken": "your.jwt.token", + "user": { + "id": "user-id", + "email": "user@example.com", + "role": "user" + } } ``` diff --git a/docs/api/dashboard.md b/docs/api/dashboard.md index 7951783..c3bc6e5 100644 --- a/docs/api/dashboard.md +++ b/docs/api/dashboard.md @@ -14,13 +14,13 @@ Retrieves overall statistics, including the total number of archived emails, tot #### Responses -- **200 OK:** An object containing the dashboard statistics. +- **200 OK:** An object containing the dashboard statistics. ```json { - "totalEmailsArchived": 12345, - "totalStorageUsed": 54321098, - "failedIngestionsLast7Days": 3 + "totalEmailsArchived": 12345, + "totalStorageUsed": 54321098, + "failedIngestionsLast7Days": 3 } ``` @@ -32,20 +32,20 @@ Retrieves the email ingestion history for the last 30 days, grouped by day. #### Responses -- **200 OK:** An object containing the ingestion history. +- **200 OK:** An object containing the ingestion history. ```json { - "history": [ - { - "date": "2023-09-27T00:00:00.000Z", - "count": 150 - }, - { - "date": "2023-09-28T00:00:00.000Z", - "count": 200 - } - ] + "history": [ + { + "date": "2023-09-27T00:00:00.000Z", + "count": 150 + }, + { + "date": "2023-09-28T00:00:00.000Z", + "count": 200 + } + ] } ``` @@ -57,24 +57,24 @@ Retrieves a list of all ingestion sources along with their status and storage us #### Responses -- **200 OK:** An array of ingestion source objects. +- **200 OK:** An array of ingestion source objects. ```json [ - { - "id": "source-id-1", - "name": "Google Workspace", - "provider": "google", - "status": "active", - "storageUsed": 12345678 - }, - { - "id": "source-id-2", - "name": "Microsoft 365", - "provider": "microsoft", - "status": "error", - "storageUsed": 87654321 - } + { + "id": "source-id-1", + "name": "Google Workspace", + "provider": "google", + "status": "active", + "storageUsed": 12345678 + }, + { + "id": "source-id-2", + "name": "Microsoft 365", + "provider": "microsoft", + "status": "error", + "storageUsed": 87654321 + } ] ``` @@ -86,7 +86,7 @@ Retrieves a list of recent synchronization jobs. (Note: This is currently a plac #### Responses -- **200 OK:** An empty array. +- **200 OK:** An empty array. ```json [] @@ -100,15 +100,15 @@ Retrieves insights from the indexed email data, such as the top senders. #### Responses -- **200 OK:** An object containing indexed insights. +- **200 OK:** An object containing indexed insights. ```json { - "topSenders": [ - { - "sender": "user@example.com", - "count": 42 - } - ] + "topSenders": [ + { + "sender": "user@example.com", + "count": 42 + } + ] } ``` diff --git a/docs/api/index.md b/docs/api/index.md index bfd8a0f..fc8ac30 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -10,9 +10,9 @@ Before making requests to protected endpoints, you must authenticate with the AP ## API Services -- [**Auth Service**](./auth.md): Handles user authentication. -- [**Archived Email Service**](./archived-email.md): Manages archived emails. -- [**Dashboard Service**](./dashboard.md): Provides data for the main dashboard. -- [**Ingestion Service**](./ingestion.md): Manages email ingestion sources. -- [**Search Service**](./search.md): Handles email search functionality. -- [**Storage Service**](./storage.md): Manages file storage and downloads. +- [**Auth Service**](./auth.md): Handles user authentication. +- [**Archived Email Service**](./archived-email.md): Manages archived emails. +- [**Dashboard Service**](./dashboard.md): Provides data for the main dashboard. +- [**Ingestion Service**](./ingestion.md): Manages email ingestion sources. +- [**Search Service**](./search.md): Handles email search functionality. +- [**Storage Service**](./storage.md): Manages file storage and downloads. diff --git a/docs/api/ingestion.md b/docs/api/ingestion.md index 247d4b2..9168d9e 100644 --- a/docs/api/ingestion.md +++ b/docs/api/ingestion.md @@ -18,16 +18,16 @@ The request body should be a `CreateIngestionSourceDto` object. ```typescript interface CreateIngestionSourceDto { - name: string; - provider: 'google' | 'microsoft' | 'generic_imap'; - providerConfig: IngestionCredentials; + name: string; + provider: 'google' | 'microsoft' | 'generic_imap'; + providerConfig: IngestionCredentials; } ``` #### Responses -- **201 Created:** The newly created ingestion source. -- **500 Internal Server Error:** An unexpected error occurred. +- **201 Created:** The newly created ingestion source. +- **500 Internal Server Error:** An unexpected error occurred. ### GET /api/v1/ingestion-sources @@ -37,8 +37,8 @@ Retrieves all ingestion sources. #### Responses -- **200 OK:** An array of ingestion source objects. -- **500 Internal Server Error:** An unexpected error occurred. +- **200 OK:** An array of ingestion source objects. +- **500 Internal Server Error:** An unexpected error occurred. ### GET /api/v1/ingestion-sources/:id @@ -54,9 +54,9 @@ Retrieves a single ingestion source by its ID. #### Responses -- **200 OK:** The ingestion source object. -- **404 Not Found:** Ingestion source not found. -- **500 Internal Server Error:** An unexpected error occurred. +- **200 OK:** The ingestion source object. +- **404 Not Found:** Ingestion source not found. +- **500 Internal Server Error:** An unexpected error occurred. ### PUT /api/v1/ingestion-sources/:id @@ -76,24 +76,18 @@ The request body should be an `UpdateIngestionSourceDto` object. ```typescript interface UpdateIngestionSourceDto { - name?: string; - provider?: 'google' | 'microsoft' | 'generic_imap'; - providerConfig?: IngestionCredentials; - status?: - | 'pending_auth' - | 'auth_success' - | 'importing' - | 'active' - | 'paused' - | 'error'; + name?: string; + provider?: 'google' | 'microsoft' | 'generic_imap'; + providerConfig?: IngestionCredentials; + status?: 'pending_auth' | 'auth_success' | 'importing' | 'active' | 'paused' | 'error'; } ``` #### Responses -- **200 OK:** The updated ingestion source object. -- **404 Not Found:** Ingestion source not found. -- **500 Internal Server Error:** An unexpected error occurred. +- **200 OK:** The updated ingestion source object. +- **404 Not Found:** Ingestion source not found. +- **500 Internal Server Error:** An unexpected error occurred. ### DELETE /api/v1/ingestion-sources/:id @@ -109,9 +103,9 @@ Deletes an ingestion source and all associated data. #### Responses -- **204 No Content:** The ingestion source was deleted successfully. -- **404 Not Found:** Ingestion source not found. -- **500 Internal Server Error:** An unexpected error occurred. +- **204 No Content:** The ingestion source was deleted successfully. +- **404 Not Found:** Ingestion source not found. +- **500 Internal Server Error:** An unexpected error occurred. ### POST /api/v1/ingestion-sources/:id/import @@ -127,9 +121,9 @@ Triggers the initial import process for an ingestion source. #### Responses -- **202 Accepted:** The initial import was triggered successfully. -- **404 Not Found:** Ingestion source not found. -- **500 Internal Server Error:** An unexpected error occurred. +- **202 Accepted:** The initial import was triggered successfully. +- **404 Not Found:** Ingestion source not found. +- **500 Internal Server Error:** An unexpected error occurred. ### POST /api/v1/ingestion-sources/:id/pause @@ -145,9 +139,9 @@ Pauses an active ingestion source. #### Responses -- **200 OK:** The updated ingestion source object with a `paused` status. -- **404 Not Found:** Ingestion source not found. -- **500 Internal Server Error:** An unexpected error occurred. +- **200 OK:** The updated ingestion source object with a `paused` status. +- **404 Not Found:** Ingestion source not found. +- **500 Internal Server Error:** An unexpected error occurred. ### POST /api/v1/ingestion-sources/:id/sync @@ -163,6 +157,6 @@ Triggers a forced synchronization for an ingestion source. #### Responses -- **202 Accepted:** The force sync was triggered successfully. -- **404 Not Found:** Ingestion source not found. -- **500 Internal Server Error:** An unexpected error occurred. +- **202 Accepted:** The force sync was triggered successfully. +- **404 Not Found:** Ingestion source not found. +- **500 Internal Server Error:** An unexpected error occurred. diff --git a/docs/api/search.md b/docs/api/search.md index 48882f5..e46d0de 100644 --- a/docs/api/search.md +++ b/docs/api/search.md @@ -24,27 +24,27 @@ Performs a search query against the indexed emails. #### Responses -- **200 OK:** A search result object. +- **200 OK:** A search result object. ```json { - "hits": [ - { - "id": "email-id", - "subject": "Test Email", - "from": "sender@example.com", - "_formatted": { - "subject": "Test Email" - } - } - ], - "total": 1, - "page": 1, - "limit": 10, - "totalPages": 1, - "processingTimeMs": 5 + "hits": [ + { + "id": "email-id", + "subject": "Test Email", + "from": "sender@example.com", + "_formatted": { + "subject": "Test Email" + } + } + ], + "total": 1, + "page": 1, + "limit": 10, + "totalPages": 1, + "processingTimeMs": 5 } ``` -- **400 Bad Request:** Keywords are required. -- **500 Internal Server Error:** An unexpected error occurred. +- **400 Bad Request:** Keywords are required. +- **500 Internal Server Error:** An unexpected error occurred. diff --git a/docs/api/storage.md b/docs/api/storage.md index b6b3e01..199c38d 100644 --- a/docs/api/storage.md +++ b/docs/api/storage.md @@ -20,7 +20,7 @@ Downloads a file from the storage. #### Responses -- **200 OK:** The file stream. -- **400 Bad Request:** File path is required or invalid. -- **404 Not Found:** File not found. -- **500 Internal Server Error:** An unexpected error occurred. +- **200 OK:** The file stream. +- **400 Bad Request:** File path is required or invalid. +- **404 Not Found:** File not found. +- **500 Internal Server Error:** An unexpected error occurred. diff --git a/docs/index.md b/docs/index.md index 78e5bcc..cd8f41c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -10,33 +10,33 @@ Open Archiver provides a robust, self-hosted solution for archiving, storing, in ## Key Features ✨ -- **Universal Ingestion**: Connect to Google Workspace, Microsoft 365, and standard IMAP servers to perform initial bulk imports and maintain continuous, real-time synchronization. -- **Secure & Efficient Storage**: Emails are stored in the standard `.eml` format. The system uses deduplication and compression to minimize storage costs. All data is encrypted at rest. -- **Pluggable Storage Backends**: Support both local filesystem storage and S3-compatible object storage (like AWS S3 or MinIO). -- **Powerful Search & eDiscovery**: A high-performance search engine indexes the full text of emails and attachments (PDF, DOCX, etc.). -- **Compliance & Retention**: Define granular retention policies to automatically manage the lifecycle of your data. Place legal holds on communications to prevent deletion during litigation (TBD). -- **Comprehensive Auditing**: An immutable audit trail logs all system activities, ensuring you have a clear record of who accessed what and when (TBD). +- **Universal Ingestion**: Connect to Google Workspace, Microsoft 365, and standard IMAP servers to perform initial bulk imports and maintain continuous, real-time synchronization. +- **Secure & Efficient Storage**: Emails are stored in the standard `.eml` format. The system uses deduplication and compression to minimize storage costs. All data is encrypted at rest. +- **Pluggable Storage Backends**: Support both local filesystem storage and S3-compatible object storage (like AWS S3 or MinIO). +- **Powerful Search & eDiscovery**: A high-performance search engine indexes the full text of emails and attachments (PDF, DOCX, etc.). +- **Compliance & Retention**: Define granular retention policies to automatically manage the lifecycle of your data. Place legal holds on communications to prevent deletion during litigation (TBD). +- **Comprehensive Auditing**: An immutable audit trail logs all system activities, ensuring you have a clear record of who accessed what and when (TBD). ## Installation 🚀 To get your own instance of Open Archiver running, follow our detailed installation guide: -- [Installation Guide](./user-guides/installation.md) +- [Installation Guide](./user-guides/installation.md) ## Data Source Configuration 🔌 After deploying the application, you will need to configure one or more ingestion sources to begin archiving emails. Follow our detailed guides to connect to your email provider: -- [Connecting to Google Workspace](./user-guides/email-providers/google-workspace.md) -- [Connecting to Microsoft 365](./user-guides/email-providers/microsoft-365.md) -- [Connecting to a Generic IMAP Server](./user-guides/email-providers/imap.md) +- [Connecting to Google Workspace](./user-guides/email-providers/google-workspace.md) +- [Connecting to Microsoft 365](./user-guides/email-providers/microsoft-365.md) +- [Connecting to a Generic IMAP Server](./user-guides/email-providers/imap.md) ## Contributing ❤️ We welcome contributions from the community! -- **Reporting Bugs**: If you find a bug, please open an issue on our GitHub repository. -- **Suggesting Enhancements**: Have an idea for a new feature? We'd love to hear it. Open an issue to start the discussion. -- **Code Contributions**: If you'd like to contribute code, please fork the repository and submit a pull request. +- **Reporting Bugs**: If you find a bug, please open an issue on our GitHub repository. +- **Suggesting Enhancements**: Have an idea for a new feature? We'd love to hear it. Open an issue to start the discussion. +- **Code Contributions**: If you'd like to contribute code, please fork the repository and submit a pull request. Please read our `CONTRIBUTING.md` file for more details on our code of conduct and the process for submitting pull requests. diff --git a/docs/services/IAM-service/iam-policies.md b/docs/services/IAM-service/iam-policies.md index 08a1891..50547b0 100644 --- a/docs/services/IAM-service/iam-policies.md +++ b/docs/services/IAM-service/iam-policies.md @@ -8,15 +8,15 @@ A policy is a JSON object that consists of one or more statements. Each statemen ```json { - "Effect": "Allow", - "Action": ["archive:read", "archive:search"], - "Resource": ["archive/all"] + "Effect": "Allow", + "Action": ["archive:read", "archive:search"], + "Resource": ["archive/all"] } ``` -- **`Effect`**: Specifies whether the statement results in an `Allow` or `Deny`. An explicit `Deny` always overrides an `Allow`. -- **`Action`**: A list of operations that the policy grants or denies permission to perform. Actions are formatted as `service:operation`. -- **`Resource`**: A list of resources to which the actions apply. Resources are specified in a hierarchical format. Wildcards (`*`) can be used. +- **`Effect`**: Specifies whether the statement results in an `Allow` or `Deny`. An explicit `Deny` always overrides an `Allow`. +- **`Action`**: A list of operations that the policy grants or denies permission to perform. Actions are formatted as `service:operation`. +- **`Resource`**: A list of resources to which the actions apply. Resources are specified in a hierarchical format. Wildcards (`*`) can be used. ## 2. Wildcard Support @@ -26,11 +26,11 @@ Our IAM system supports wildcards (`*`) in both `Action` and `Resource` fields t You can use wildcards to grant broad permissions for actions: -- **Global Wildcard (`*`)**: A standalone `*` in the `Action` field grants permission for all possible actions across all services. +- **Global Wildcard (`*`)**: A standalone `*` in the `Action` field grants permission for all possible actions across all services. ```json "Action": ["*"] ``` -- **Service-Level Wildcard (`service:*`)**: A wildcard at the end of an action string grants permission for all actions within that specific service. +- **Service-Level Wildcard (`service:*`)**: A wildcard at the end of an action string grants permission for all actions within that specific service. ```json "Action": ["archive:*"] ``` @@ -39,11 +39,11 @@ You can use wildcards to grant broad permissions for actions: Wildcards can also be used to specify resources: -- **Global Wildcard (`*`)**: A standalone `*` in the `Resource` field applies the policy to all resources in the system. +- **Global Wildcard (`*`)**: A standalone `*` in the `Resource` field applies the policy to all resources in the system. ```json "Resource": ["*"] ``` -- **Partial Wildcards**: Some services allow wildcards at specific points in the resource path to refer to all resources of a certain type. For example, to target all ingestion sources: +- **Partial Wildcards**: Some services allow wildcards at specific points in the resource path to refer to all resources of a certain type. For example, to target all ingestion sources: ```json "Resource": ["ingestion-source/*"] ``` diff --git a/docs/services/index.md b/docs/services/index.md index 34ce7e5..a0b12d7 100644 --- a/docs/services/index.md +++ b/docs/services/index.md @@ -1,2 +1 @@ # services - diff --git a/docs/services/storage-service.md b/docs/services/storage-service.md index 92c9573..461d945 100644 --- a/docs/services/storage-service.md +++ b/docs/services/storage-service.md @@ -14,8 +14,8 @@ The `StorageService` is configured via environment variables in the `.env` file. The `STORAGE_TYPE` variable determines which provider the service will use. -- `STORAGE_TYPE=local`: Uses the local server's filesystem. -- `STORAGE_TYPE=s3`: Uses an S3-compatible object storage service (e.g., AWS S3, MinIO, Google Cloud Storage). +- `STORAGE_TYPE=local`: Uses the local server's filesystem. +- `STORAGE_TYPE=s3`: Uses an S3-compatible object storage service (e.g., AWS S3, MinIO, Google Cloud Storage). ### 2. Local Filesystem Configuration @@ -27,7 +27,7 @@ STORAGE_TYPE=local STORAGE_LOCAL_ROOT_PATH=/var/data/open-archiver ``` -- `STORAGE_LOCAL_ROOT_PATH`: The absolute path on the server where the archive will be created. The service will create subdirectories within this path as needed. +- `STORAGE_LOCAL_ROOT_PATH`: The absolute path on the server where the archive will be created. The service will create subdirectories within this path as needed. ### 3. S3-Compatible Storage Configuration @@ -44,12 +44,12 @@ STORAGE_S3_REGION=us-east-1 STORAGE_S3_FORCE_PATH_STYLE=true ``` -- `STORAGE_S3_ENDPOINT`: The full URL of the S3 API endpoint. -- `STORAGE_S3_BUCKET`: The name of the bucket to use for storage. -- `STORAGE_S3_ACCESS_KEY_ID`: The access key for your S3 user. -- `STORAGE_S3_SECRET_ACCESS_KEY`: The secret key for your S3 user. -- `STORAGE_S3_REGION` (Optional): The AWS region of your bucket. Recommended for AWS S3. -- `STORAGE_S3_FORCE_PATH_STYLE` (Optional): Set to `true` when using non-AWS S3 services like MinIO. +- `STORAGE_S3_ENDPOINT`: The full URL of the S3 API endpoint. +- `STORAGE_S3_BUCKET`: The name of the bucket to use for storage. +- `STORAGE_S3_ACCESS_KEY_ID`: The access key for your S3 user. +- `STORAGE_S3_SECRET_ACCESS_KEY`: The secret key for your S3 user. +- `STORAGE_S3_REGION` (Optional): The AWS region of your bucket. Recommended for AWS S3. +- `STORAGE_S3_FORCE_PATH_STYLE` (Optional): Set to `true` when using non-AWS S3 services like MinIO. ## How to Use the Service @@ -61,31 +61,27 @@ The `StorageService` is designed to be used via dependency injection in other se import { StorageService } from './StorageService'; class IngestionService { - private storageService: StorageService; + private storageService: StorageService; - constructor() { - // The StorageService is instantiated without any arguments. - // It automatically reads the configuration from the environment. - this.storageService = new StorageService(); - } + constructor() { + // The StorageService is instantiated without any arguments. + // It automatically reads the configuration from the environment. + this.storageService = new StorageService(); + } - public async archiveEmail( - rawEmail: Buffer, - userId: string, - messageId: string - ): Promise { - // Define a structured, unique path for the email. - const archivePath = `${userId}/messages/${messageId}.eml`; + public async archiveEmail(rawEmail: Buffer, userId: string, messageId: string): Promise { + // Define a structured, unique path for the email. + const archivePath = `${userId}/messages/${messageId}.eml`; - try { - // Use the service. It doesn't know or care if this is writing - // to a local disk or an S3 bucket. - await this.storageService.put(archivePath, rawEmail); - console.log(`Successfully archived email to ${archivePath}`); - } catch (error) { - console.error(`Failed to archive email ${messageId}`, error); - } - } + try { + // Use the service. It doesn't know or care if this is writing + // to a local disk or an S3 bucket. + await this.storageService.put(archivePath, rawEmail); + console.log(`Successfully archived email to ${archivePath}`); + } catch (error) { + console.error(`Failed to archive email ${messageId}`, error); + } + } } ``` @@ -99,9 +95,9 @@ The `StorageService` implements the `IStorageProvider` interface. All methods ar Stores a file at the specified path. If a file already exists at that path, it will be overwritten. -- **`path: string`**: A unique identifier for the file, including its directory structure (e.g., `"user-123/emails/message-abc.eml"`). -- **`content: Buffer | NodeJS.ReadableStream`**: The content of the file. It can be a `Buffer` for small files or a `ReadableStream` for large files to ensure memory efficiency. -- **Returns**: `Promise` - A promise that resolves when the file has been successfully stored. +- **`path: string`**: A unique identifier for the file, including its directory structure (e.g., `"user-123/emails/message-abc.eml"`). +- **`content: Buffer | NodeJS.ReadableStream`**: The content of the file. It can be a `Buffer` for small files or a `ReadableStream` for large files to ensure memory efficiency. +- **Returns**: `Promise` - A promise that resolves when the file has been successfully stored. --- @@ -109,9 +105,9 @@ Stores a file at the specified path. If a file already exists at that path, it w Retrieves a file from the specified path as a readable stream. -- **`path: string`**: The unique identifier of the file to retrieve. -- **Returns**: `Promise` - A promise that resolves with a readable stream of the file's content. -- **Throws**: An `Error` if the file is not found at the specified path. +- **`path: string`**: The unique identifier of the file to retrieve. +- **Returns**: `Promise` - A promise that resolves with a readable stream of the file's content. +- **Throws**: An `Error` if the file is not found at the specified path. --- @@ -119,8 +115,8 @@ Retrieves a file from the specified path as a readable stream. Deletes a file from the storage backend. -- **`path: string`**: The unique identifier of the file to delete. -- **Returns**: `Promise` - A promise that resolves when the file is deleted. If the file does not exist, the promise will still resolve successfully without throwing an error. +- **`path: string`**: The unique identifier of the file to delete. +- **Returns**: `Promise` - A promise that resolves when the file is deleted. If the file does not exist, the promise will still resolve successfully without throwing an error. --- @@ -128,5 +124,5 @@ Deletes a file from the storage backend. Checks for the existence of a file. -- **`path: string`**: The unique identifier of the file to check. -- **Returns**: `Promise` - A promise that resolves with `true` if the file exists, and `false` otherwise. +- **`path: string`**: The unique identifier of the file to check. +- **Returns**: `Promise` - A promise that resolves with `true` if the file exists, and `false` otherwise. diff --git a/docs/user-guides/email-providers/eml.md b/docs/user-guides/email-providers/eml.md index e151b1b..157cb35 100644 --- a/docs/user-guides/email-providers/eml.md +++ b/docs/user-guides/email-providers/eml.md @@ -6,8 +6,8 @@ OpenArchiver allows you to import EML files from a zip archive. This is useful f To ensure a successful import, you should compress your .eml files to one zip file according to the following guidelines: -- **Structure:** The zip file can contain any number of `.eml` files, organized in any folder structure. The folder structure will be preserved in OpenArchiver, so you can use it to organize your emails. -- **Compression:** The zip file should be compressed using standard zip compression. +- **Structure:** The zip file can contain any number of `.eml` files, organized in any folder structure. The folder structure will be preserved in OpenArchiver, so you can use it to organize your emails. +- **Compression:** The zip file should be compressed using standard zip compression. Here's an example of a valid folder structure: diff --git a/docs/user-guides/email-providers/google-workspace.md b/docs/user-guides/email-providers/google-workspace.md index 0f04539..e77fc22 100644 --- a/docs/user-guides/email-providers/google-workspace.md +++ b/docs/user-guides/email-providers/google-workspace.md @@ -6,8 +6,8 @@ The connection uses a **Google Cloud Service Account** with **Domain-Wide Delega ## Prerequisites -- You must have **Super Administrator** privileges in your Google Workspace account. -- You must have access to the **Google Cloud Console** associated with your organization. +- You must have **Super Administrator** privileges in your Google Workspace account. +- You must have access to the **Google Cloud Console** associated with your organization. ## Setup Overview @@ -24,30 +24,27 @@ The setup process involves three main parts: In this part, you will create a service account and enable the APIs it needs to function. 1. **Create a Google Cloud Project:** - - - Go to the [Google Cloud Console](https://console.cloud.google.com/). - - If you don't already have one, create a new project for the archiving service (e.g., "Email Archiver"). + - Go to the [Google Cloud Console](https://console.cloud.google.com/). + - If you don't already have one, create a new project for the archiving service (e.g., "Email Archiver"). 2. **Enable Required APIs:** - - - In your selected project, navigate to the **"APIs & Services" > "Library"** section. - - Search for and enable the following two APIs: - - **Gmail API** - - **Admin SDK API** + - In your selected project, navigate to the **"APIs & Services" > "Library"** section. + - Search for and enable the following two APIs: + - **Gmail API** + - **Admin SDK API** 3. **Create a Service Account:** - - - Navigate to **"IAM & Admin" > "Service Accounts"**. - - Click **"Create Service Account"**. - - Give the service account a name (e.g., `email-archiver-service`) and a description. - - Click **"Create and Continue"**. You do not need to grant this service account any roles on the project. Click **"Done"**. + - Navigate to **"IAM & Admin" > "Service Accounts"**. + - Click **"Create Service Account"**. + - Give the service account a name (e.g., `email-archiver-service`) and a description. + - Click **"Create and Continue"**. You do not need to grant this service account any roles on the project. Click **"Done"**. 4. **Generate a JSON Key:** - - Find the service account you just created in the list. - - Click the three-dot menu under **"Actions"** and select **"Manage keys"**. - - Click **"Add Key"** > **"Create new key"**. - - Select **JSON** as the key type and click **"Create"**. - - A JSON file will be downloaded to your computer. **Keep this file secure, as it contains private credentials.** You will need the contents of this file in Part 3. + - Find the service account you just created in the list. + - Click the three-dot menu under **"Actions"** and select **"Manage keys"**. + - Click **"Add Key"** > **"Create new key"**. + - Select **JSON** as the key type and click **"Create"**. + - A JSON file will be downloaded to your computer. **Keep this file secure, as it contains private credentials.** You will need the contents of this file in Part 3. ### Troubleshooting @@ -60,14 +57,14 @@ To resolve this, you must have **Organization Administrator** permissions. 1. **Navigate to your Organization:** In the Google Cloud Console, use the project selector at the top of the page to select your organization node (it usually has a building icon). 2. **Go to IAM:** From the navigation menu, select **"IAM & Admin" > "IAM"**. 3. **Edit Your Permissions:** Find your user account in the list and click the pencil icon to edit roles. Add the following two roles: - - `Organization Policy Administrator` - - `Organization Administrator` - _Note: These roles are only available at the organization level, not the project level._ + - `Organization Policy Administrator` + - `Organization Administrator` + _Note: These roles are only available at the organization level, not the project level._ 4. **Modify the Policy:** - - Navigate to **"IAM & Admin" > "Organization Policies"**. - - In the filter box, search for the policy **"iam.disableServiceAccountKeyCreation"**. - - Click on the policy to edit it. - - You can either disable the policy entirely (if your security rules permit) or add a rule to exclude the specific project you are using for the archiver from this policy. + - Navigate to **"IAM & Admin" > "Organization Policies"**. + - In the filter box, search for the policy **"iam.disableServiceAccountKeyCreation"**. + - Click on the policy to edit it. + - You can either disable the policy entirely (if your security rules permit) or add a rule to exclude the specific project you are using for the archiver from this policy. 5. **Retry Key Creation:** Once the policy is updated, return to your project and you should be able to generate the JSON key as described in Part 1. --- @@ -77,25 +74,23 @@ To resolve this, you must have **Organization Administrator** permissions. Now, you will authorize the service account you created to access data from your Google Workspace. 1. **Get the Service Account's Client ID:** - - - Go back to the list of service accounts in the Google Cloud Console. - - Click on the service account you created. - - Under the **"Details"** tab, find and copy the **Unique ID** (this is the Client ID). + - Go back to the list of service accounts in the Google Cloud Console. + - Click on the service account you created. + - Under the **"Details"** tab, find and copy the **Unique ID** (this is the Client ID). 2. **Authorize the Client in Google Workspace:** - - - Go to your **Google Workspace Admin Console** at [admin.google.com](https://admin.google.com). - - Navigate to **Security > Access and data control > API controls**. - - Under the "Domain-wide Delegation" section, click **"Manage Domain-wide Delegation"**. - - Click **"Add new"**. + - Go to your **Google Workspace Admin Console** at [admin.google.com](https://admin.google.com). + - Navigate to **Security > Access and data control > API controls**. + - Under the "Domain-wide Delegation" section, click **"Manage Domain-wide Delegation"**. + - Click **"Add new"**. 3. **Enter Client Details and Scopes:** - - In the **Client ID** field, paste the **Unique ID** you copied from the service account. - - In the **OAuth scopes** field, paste the following two scopes exactly as they appear, separated by a comma: + - In the **Client ID** field, paste the **Unique ID** you copied from the service account. + - In the **OAuth scopes** field, paste the following two scopes exactly as they appear, separated by a comma: ``` https://www.googleapis.com/auth/admin.directory.user.readonly,https://www.googleapis.com/auth/gmail.readonly ``` - - Click **"Authorize"**. + - Click **"Authorize"**. The service account is now permitted to list users and read their email data across your domain. @@ -112,11 +107,10 @@ Finally, you will provide the generated credentials to the application. Click the **"Create New"** button. 3. **Fill in the Configuration Details:** - - - **Name:** Give the source a name (e.g., "Google Workspace Archive"). - - **Provider:** Select **"Google Workspace"** from the dropdown. - - **Service Account Key (JSON):** Open the JSON file you downloaded in Part 1. Copy the entire content of the file and paste it into this text area. - - **Impersonated Admin Email:** Enter the email address of a Super Administrator in your Google Workspace (e.g., `admin@your-domain.com`). The service will use this user's authority to discover all other users. + - **Name:** Give the source a name (e.g., "Google Workspace Archive"). + - **Provider:** Select **"Google Workspace"** from the dropdown. + - **Service Account Key (JSON):** Open the JSON file you downloaded in Part 1. Copy the entire content of the file and paste it into this text area. + - **Impersonated Admin Email:** Enter the email address of a Super Administrator in your Google Workspace (e.g., `admin@your-domain.com`). The service will use this user's authority to discover all other users. 4. **Save Changes:** Click **"Save changes"**. diff --git a/docs/user-guides/email-providers/imap.md b/docs/user-guides/email-providers/imap.md index 11c8b02..40ec171 100644 --- a/docs/user-guides/email-providers/imap.md +++ b/docs/user-guides/email-providers/imap.md @@ -12,18 +12,17 @@ This guide will walk you through connecting a standard IMAP email account as an 3. **Fill in the Configuration Details:** You will see a form with several fields. Here is how to fill them out for an IMAP connection: + - **Name:** Give your ingestion source a descriptive name that you will easily recognize, such as "Work Email (IMAP)" or "Personal Gmail". - - **Name:** Give your ingestion source a descriptive name that you will easily recognize, such as "Work Email (IMAP)" or "Personal Gmail". + - **Provider:** From the dropdown menu, select **"Generic IMAP"**. This will reveal the specific fields required for an IMAP connection. - - **Provider:** From the dropdown menu, select **"Generic IMAP"**. This will reveal the specific fields required for an IMAP connection. + - **Host:** Enter the server address for your email provider's IMAP service. This often looks like `imap.your-provider.com` or `mail.your-domain.com`. - - **Host:** Enter the server address for your email provider's IMAP service. This often looks like `imap.your-provider.com` or `mail.your-domain.com`. + - **Port:** Enter the port number for the IMAP server. For a secure connection (which is strongly recommended), this is typically `993`. - - **Port:** Enter the port number for the IMAP server. For a secure connection (which is strongly recommended), this is typically `993`. + - **Username:** Enter the full email address or username you use to log in to your email account. - - **Username:** Enter the full email address or username you use to log in to your email account. - - - **Password:** Enter the password for your email account. + - **Password:** Enter the password for your email account. 4. **Save Changes:** Once you have filled in all the details, click the **"Save changes"** button. @@ -41,9 +40,9 @@ Please consult your email provider's documentation to see if they support app pa 1. **Enable 2-Step Verification:** You must have 2-Step Verification turned on for your Google Account. 2. **Go to App Passwords:** Visit [myaccount.google.com/apppasswords](https://myaccount.google.com/apppasswords). You may be asked to sign in again. 3. **Create the Password:** - - At the bottom, click **"Select app"** and choose **"Other (Custom name)"**. - - Give it a name you'll recognize, like "OpenArchiver". - - Click **"Generate"**. + - At the bottom, click **"Select app"** and choose **"Other (Custom name)"**. + - Give it a name you'll recognize, like "OpenArchiver". + - Click **"Generate"**. 4. **Use the Password:** A 16-digit password will be displayed. Copy this password and paste it into the **Password** field in the OpenArchiver ingestion source form. ### How to Obtain an App Password for Outlook/Microsoft Accounts @@ -51,17 +50,17 @@ Please consult your email provider's documentation to see if they support app pa 1. **Enable Two-Step Verification:** You must have two-step verification enabled for your Microsoft account. 2. **Go to Security Options:** Sign in to your Microsoft account and navigate to the [Advanced security options](https://account.live.com/proofs/manage/additional). 3. **Create a New App Password:** - - Scroll down to the **"App passwords"** section. - - Click **"Create a new app password"**. + - Scroll down to the **"App passwords"** section. + - Click **"Create a new app password"**. 4. **Use the Password:** A new password will be generated. Use this password in the **Password** field in the OpenArchiver ingestion source form. ## What Happens Next? After you save the connection, the system will attempt to connect to the IMAP server. The status of the ingestion source will update to reflect its current state: -- **Importing:** The system is performing the initial, one-time import of all emails from your `INBOX`. This may take a while depending on the size of your mailbox. -- **Active:** The initial import is complete, and the system will now periodically check for and archive new emails. -- **Paused:** The connection is valid, but the system will not check for new emails until you resume it. -- **Error:** The system was unable to connect using the provided credentials. Please double-check your Host, Port, Username, and Password and try again. +- **Importing:** The system is performing the initial, one-time import of all emails from your `INBOX`. This may take a while depending on the size of your mailbox. +- **Active:** The initial import is complete, and the system will now periodically check for and archive new emails. +- **Paused:** The connection is valid, but the system will not check for new emails until you resume it. +- **Error:** The system was unable to connect using the provided credentials. Please double-check your Host, Port, Username, and Password and try again. You can view, edit, pause, or manually sync any of your ingestion sources from the main table on the **Ingestions** page. diff --git a/docs/user-guides/email-providers/index.md b/docs/user-guides/email-providers/index.md index dfa8afd..1b1c8f7 100644 --- a/docs/user-guides/email-providers/index.md +++ b/docs/user-guides/email-providers/index.md @@ -4,8 +4,8 @@ Open Archiver can connect to a variety of email sources to ingest and archive yo Choose your provider from the list below to get started: -- [Google Workspace](./google-workspace.md) -- [Microsoft 365](./microsoft-365.md) -- [Generic IMAP Server](./imap.md) -- [EML Import](./eml.md) -- [PST Import](./pst.md) +- [Google Workspace](./google-workspace.md) +- [Microsoft 365](./microsoft-365.md) +- [Generic IMAP Server](./imap.md) +- [EML Import](./eml.md) +- [PST Import](./pst.md) diff --git a/docs/user-guides/email-providers/microsoft-365.md b/docs/user-guides/email-providers/microsoft-365.md index d8c3c4b..53c6293 100644 --- a/docs/user-guides/email-providers/microsoft-365.md +++ b/docs/user-guides/email-providers/microsoft-365.md @@ -6,7 +6,7 @@ The connection uses the **Microsoft Graph API** and an **App Registration** in M ## Prerequisites -- You must have one of the following administrator roles in your Microsoft 365 tenant: **Global Administrator**, **Application Administrator**, or **Cloud Application Administrator**. +- You must have one of the following administrator roles in your Microsoft 365 tenant: **Global Administrator**, **Application Administrator**, or **Cloud Application Administrator**. ## Setup Overview @@ -27,9 +27,9 @@ First, you will create an "App registration," which acts as an identity for the 2. In the left-hand navigation pane, go to **Identity > Applications > App registrations**. 3. Click the **+ New registration** button at the top of the page. 4. On the "Register an application" screen: - - **Name:** Give the application a descriptive name you will recognize, such as `OpenArchiver Service`. - - **Supported account types:** Select **"Accounts in this organizational directory only (Default Directory only - Single tenant)"**. This is the most secure option. - - **Redirect URI (optional):** You can leave this blank. + - **Name:** Give the application a descriptive name you will recognize, such as `OpenArchiver Service`. + - **Supported account types:** Select **"Accounts in this organizational directory only (Default Directory only - Single tenant)"**. This is the most secure option. + - **Redirect URI (optional):** You can leave this blank. 5. Click the **Register** button. You will be taken to the application's main "Overview" page. --- @@ -43,8 +43,8 @@ Next, you must grant the application the specific permissions required to read u 3. In the "Request API permissions" pane, select **Microsoft Graph**. 4. Select **Application permissions**. This is critical as it allows the service to run in the background without a user being signed in. 5. In the "Select permissions" search box, find and check the boxes for the following two permissions: - - `Mail.Read` - - `User.Read.All` + - `Mail.Read` + - `User.Read.All` 6. Click the **Add permissions** button at the bottom. 7. **Crucial Final Step:** You will now see the permissions in your list with a warning status. You must grant consent on behalf of your organization. Click the **"Grant admin consent for [Your Organization's Name]"** button located above the permissions table. Click **Yes** in the confirmation dialog. The status for both permissions should now show a green checkmark. @@ -57,8 +57,8 @@ The client secret is a password that the archiving service will use to authentic 1. In your application's menu, navigate to **Certificates & secrets**. 2. Select the **Client secrets** tab and click **+ New client secret**. 3. In the pane that appears: - - **Description:** Enter a clear description, such as `OpenArchiver Key`. - - **Expires:** Select an expiry duration. We recommend **12 or 24 months**. Set a calendar reminder to renew it before it expires to prevent service interruption. + - **Description:** Enter a clear description, such as `OpenArchiver Key`. + - **Expires:** Select an expiry duration. We recommend **12 or 24 months**. Set a calendar reminder to renew it before it expires to prevent service interruption. 4. Click **Add**. 5. **IMMEDIATELY COPY THE SECRET:** The secret is now visible in the **"Value"** column. This is the only time it will be fully displayed. Copy this value now and store it in a secure password manager before navigating away. If you lose it, you must create a new one. @@ -75,12 +75,11 @@ You now have the three pieces of information required to configure the connectio Click the **"Create New"** button. 3. **Fill in the Configuration Details:** - - - **Name:** Give the source a name (e.g., "Microsoft 365 Archive"). - - **Provider:** Select **"Microsoft 365"** from the dropdown. - - **Application (Client) ID:** Go to the **Overview** page of your app registration in the Entra admin center and copy this value. - - **Directory (Tenant) ID:** This value is also on the **Overview** page. - - **Client Secret Value:** Paste the secret **Value** (not the Secret ID) that you copied and saved in the previous step. + - **Name:** Give the source a name (e.g., "Microsoft 365 Archive"). + - **Provider:** Select **"Microsoft 365"** from the dropdown. + - **Application (Client) ID:** Go to the **Overview** page of your app registration in the Entra admin center and copy this value. + - **Directory (Tenant) ID:** This value is also on the **Overview** page. + - **Client Secret Value:** Paste the secret **Value** (not the Secret ID) that you copied and saved in the previous step. 4. **Save Changes:** Click **"Save changes"**. diff --git a/docs/user-guides/email-providers/pst.md b/docs/user-guides/email-providers/pst.md index a029450..4180daa 100644 --- a/docs/user-guides/email-providers/pst.md +++ b/docs/user-guides/email-providers/pst.md @@ -6,8 +6,8 @@ OpenArchiver allows you to import PST files. This is useful for importing emails To ensure a successful import, you should prepare your PST file according to the following guidelines: -- **Structure:** The PST file can contain any number of emails, organized in any folder structure. The folder structure will be preserved in OpenArchiver, so you can use it to organize your emails. -- **Password Protection:** OpenArchiver does not support password-protected PST files. Please remove the password from your PST file before importing it. +- **Structure:** The PST file can contain any number of emails, organized in any folder structure. The folder structure will be preserved in OpenArchiver, so you can use it to organize your emails. +- **Password Protection:** OpenArchiver does not support password-protected PST files. Please remove the password from your PST file before importing it. ## Creating a PST Ingestion Source diff --git a/docs/user-guides/installation.md b/docs/user-guides/installation.md index 7b26955..0a40cd3 100644 --- a/docs/user-guides/installation.md +++ b/docs/user-guides/installation.md @@ -4,9 +4,9 @@ This guide will walk you through setting up Open Archiver using Docker Compose. ## Prerequisites -- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your server or local machine. -- A server or local machine with at least 4GB of RAM (2GB of RAM if you use external Postgres, Redis (Valkey) and Meilisearch instances). -- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) installed on your server or local machine. +- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your server or local machine. +- A server or local machine with at least 4GB of RAM (2GB of RAM if you use external Postgres, Redis (Valkey) and Meilisearch instances). +- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) installed on your server or local machine. ## 1. Clone the Repository @@ -33,11 +33,11 @@ Now, open the `.env` file in a text editor and customize the settings. You must change the following placeholder values to secure your instance: -- `POSTGRES_PASSWORD`: A strong, unique password for the database. -- `REDIS_PASSWORD`: A strong, unique password for the Valkey/Redis service. -- `MEILI_MASTER_KEY`: A complex key for Meilisearch. -- `JWT_SECRET`: A long, random string for signing authentication tokens. -- `ENCRYPTION_KEY`: A 32-byte hex string for encrypting sensitive data in the database. You can generate one with the following command: +- `POSTGRES_PASSWORD`: A strong, unique password for the database. +- `REDIS_PASSWORD`: A strong, unique password for the Valkey/Redis service. +- `MEILI_MASTER_KEY`: A complex key for Meilisearch. +- `JWT_SECRET`: A long, random string for signing authentication tokens. +- `ENCRYPTION_KEY`: A 32-byte hex string for encrypting sensitive data in the database. You can generate one with the following command: ```bash openssl rand -hex 32 ``` @@ -122,9 +122,9 @@ docker compose up -d This command will: -- Pull the required Docker images for the frontend, backend, database, and other services. -- Create and start the containers in the background (`-d` flag). -- Create the persistent volumes for your data. +- Pull the required Docker images for the frontend, backend, database, and other services. +- Create and start the containers in the background (`-d` flag). +- Create the persistent volumes for your data. You can check the status of the running containers with: @@ -142,9 +142,9 @@ You can log in with the `ADMIN_EMAIL` and `ADMIN_PASSWORD` you configured in you After successfully deploying and logging into Open Archiver, the next step is to configure your ingestion sources to start archiving emails. -- [Connecting to Google Workspace](./email-providers/google-workspace.md) -- [Connecting to Microsoft 365](./email-providers/microsoft-365.md) -- [Connecting to a Generic IMAP Server](./email-providers/imap.md) +- [Connecting to Google Workspace](./email-providers/google-workspace.md) +- [Connecting to Microsoft 365](./email-providers/microsoft-365.md) +- [Connecting to a Generic IMAP Server](./email-providers/imap.md) ## Updating Your Installation @@ -174,9 +174,8 @@ To do this, you will need to make a small modification to your `docker-compose.y 2. **Remove all `networks` sections** from the file. This includes the network configuration for each service and the top-level network definition. Specifically, you need to remove: - - - The `networks: - open-archiver-net` lines from the `open-archiver`, `postgres`, `valkey`, and `meilisearch` services. - - The entire `networks:` block at the end of the file. + - The `networks: - open-archiver-net` lines from the `open-archiver`, `postgres`, `valkey`, and `meilisearch` services. + - The entire `networks:` block at the end of the file. Here is an example of what to remove from a service: diff --git a/package.json b/package.json index 0769d54..355167a 100644 --- a/package.json +++ b/package.json @@ -14,7 +14,7 @@ "docs:dev": "vitepress dev docs --port 3009", "docs:build": "vitepress build docs", "docs:preview": "vitepress preview docs", - "format": "prettier --write .", + "format": "prettier --write .", "lint": "prettier --check ." }, "dependencies": { @@ -23,7 +23,7 @@ }, "devDependencies": { "prettier": "^3.6.2", - "prettier-plugin-svelte": "^3.4.0", + "prettier-plugin-svelte": "^3.4.0", "prettier-plugin-tailwindcss": "^0.6.14", "typescript": "5.8.3", "vitepress": "^1.6.4" diff --git a/packages/backend/drizzle.config.ts b/packages/backend/drizzle.config.ts index 58336ac..82c1132 100644 --- a/packages/backend/drizzle.config.ts +++ b/packages/backend/drizzle.config.ts @@ -4,16 +4,16 @@ import { config } from 'dotenv'; config(); if (!process.env.DATABASE_URL) { - throw new Error('DATABASE_URL is not set in the .env file'); + throw new Error('DATABASE_URL is not set in the .env file'); } export default defineConfig({ - schema: './src/database/schema.ts', - out: './src/database/migrations', - dialect: 'postgresql', - dbCredentials: { - url: process.env.DATABASE_URL, - }, - verbose: true, - strict: true, + schema: './src/database/schema.ts', + out: './src/database/migrations', + dialect: 'postgresql', + dbCredentials: { + url: process.env.DATABASE_URL, + }, + verbose: true, + strict: true, }); diff --git a/packages/backend/package.json b/packages/backend/package.json index 47693e6..76929af 100644 --- a/packages/backend/package.json +++ b/packages/backend/package.json @@ -1,75 +1,75 @@ { - "name": "@open-archiver/backend", - "version": "0.1.0", - "private": true, - "main": "dist/index.js", - "scripts": { - "dev": "ts-node-dev --respawn --transpile-only src/index.ts ", - "build": "tsc", - "start": "node dist/index.js", - "start:ingestion-worker": "node dist/workers/ingestion.worker.js", - "start:indexing-worker": "node dist/workers/indexing.worker.js", - "start:sync-scheduler": "node dist/jobs/schedulers/sync-scheduler.js", - "start:ingestion-worker:dev": "ts-node-dev --respawn --transpile-only src/workers/ingestion.worker.ts", - "start:indexing-worker:dev": "ts-node-dev --respawn --transpile-only src/workers/indexing.worker.ts", - "start:sync-scheduler:dev": "ts-node-dev --respawn --transpile-only src/jobs/schedulers/sync-scheduler.ts", - "db:generate": "drizzle-kit generate --config=drizzle.config.ts", - "db:push": "drizzle-kit push --config=drizzle.config.ts", - "db:migrate": "node dist/database/migrate.js", - "db:migrate:dev": "ts-node-dev src/database/migrate.ts" - }, - "dependencies": { - "@aws-sdk/client-s3": "^3.844.0", - "@aws-sdk/lib-storage": "^3.844.0", - "@azure/msal-node": "^3.6.3", - "@microsoft/microsoft-graph-client": "^3.0.7", - "@open-archiver/types": "workspace:*", - "archiver": "^7.0.1", - "axios": "^1.10.0", - "bcryptjs": "^3.0.2", - "bullmq": "^5.56.3", - "busboy": "^1.6.0", - "cross-fetch": "^4.1.0", - "deepmerge-ts": "^7.1.5", - "dotenv": "^17.2.0", - "drizzle-kit": "^0.31.4", - "drizzle-orm": "^0.44.2", - "express": "^5.1.0", - "express-rate-limit": "^8.0.1", - "express-validator": "^7.2.1", - "google-auth-library": "^10.1.0", - "googleapis": "^152.0.0", - "imapflow": "^1.0.191", - "jose": "^6.0.11", - "mailparser": "^3.7.4", - "mammoth": "^1.9.1", - "meilisearch": "^0.51.0", - "multer": "^2.0.2", - "pdf2json": "^3.1.6", - "pg": "^8.16.3", - "pino": "^9.7.0", - "pino-pretty": "^13.0.0", - "postgres": "^3.4.7", - "pst-extractor": "^1.11.0", - "reflect-metadata": "^0.2.2", - "sqlite3": "^5.1.7", - "tsconfig-paths": "^4.2.0", - "xlsx": "^0.18.5", - "yauzl": "^3.2.0" - }, - "devDependencies": { - "@bull-board/api": "^6.11.0", - "@bull-board/express": "^6.11.0", - "@types/archiver": "^6.0.3", - "@types/busboy": "^1.5.4", - "@types/express": "^5.0.3", - "@types/mailparser": "^3.4.6", - "@types/microsoft-graph": "^2.40.1", - "@types/multer": "^2.0.0", - "@types/node": "^24.0.12", - "@types/yauzl": "^2.10.3", - "bull-board": "^2.1.3", - "ts-node-dev": "^2.0.0", - "typescript": "^5.8.3" - } + "name": "@open-archiver/backend", + "version": "0.1.0", + "private": true, + "main": "dist/index.js", + "scripts": { + "dev": "ts-node-dev --respawn --transpile-only src/index.ts ", + "build": "tsc", + "start": "node dist/index.js", + "start:ingestion-worker": "node dist/workers/ingestion.worker.js", + "start:indexing-worker": "node dist/workers/indexing.worker.js", + "start:sync-scheduler": "node dist/jobs/schedulers/sync-scheduler.js", + "start:ingestion-worker:dev": "ts-node-dev --respawn --transpile-only src/workers/ingestion.worker.ts", + "start:indexing-worker:dev": "ts-node-dev --respawn --transpile-only src/workers/indexing.worker.ts", + "start:sync-scheduler:dev": "ts-node-dev --respawn --transpile-only src/jobs/schedulers/sync-scheduler.ts", + "db:generate": "drizzle-kit generate --config=drizzle.config.ts", + "db:push": "drizzle-kit push --config=drizzle.config.ts", + "db:migrate": "node dist/database/migrate.js", + "db:migrate:dev": "ts-node-dev src/database/migrate.ts" + }, + "dependencies": { + "@aws-sdk/client-s3": "^3.844.0", + "@aws-sdk/lib-storage": "^3.844.0", + "@azure/msal-node": "^3.6.3", + "@microsoft/microsoft-graph-client": "^3.0.7", + "@open-archiver/types": "workspace:*", + "archiver": "^7.0.1", + "axios": "^1.10.0", + "bcryptjs": "^3.0.2", + "bullmq": "^5.56.3", + "busboy": "^1.6.0", + "cross-fetch": "^4.1.0", + "deepmerge-ts": "^7.1.5", + "dotenv": "^17.2.0", + "drizzle-kit": "^0.31.4", + "drizzle-orm": "^0.44.2", + "express": "^5.1.0", + "express-rate-limit": "^8.0.1", + "express-validator": "^7.2.1", + "google-auth-library": "^10.1.0", + "googleapis": "^152.0.0", + "imapflow": "^1.0.191", + "jose": "^6.0.11", + "mailparser": "^3.7.4", + "mammoth": "^1.9.1", + "meilisearch": "^0.51.0", + "multer": "^2.0.2", + "pdf2json": "^3.1.6", + "pg": "^8.16.3", + "pino": "^9.7.0", + "pino-pretty": "^13.0.0", + "postgres": "^3.4.7", + "pst-extractor": "^1.11.0", + "reflect-metadata": "^0.2.2", + "sqlite3": "^5.1.7", + "tsconfig-paths": "^4.2.0", + "xlsx": "^0.18.5", + "yauzl": "^3.2.0" + }, + "devDependencies": { + "@bull-board/api": "^6.11.0", + "@bull-board/express": "^6.11.0", + "@types/archiver": "^6.0.3", + "@types/busboy": "^1.5.4", + "@types/express": "^5.0.3", + "@types/mailparser": "^3.4.6", + "@types/microsoft-graph": "^2.40.1", + "@types/multer": "^2.0.0", + "@types/node": "^24.0.12", + "@types/yauzl": "^2.10.3", + "bull-board": "^2.1.3", + "ts-node-dev": "^2.0.0", + "typescript": "^5.8.3" + } } diff --git a/packages/backend/src/api/controllers/archived-email.controller.ts b/packages/backend/src/api/controllers/archived-email.controller.ts index 3c2b704..648deec 100644 --- a/packages/backend/src/api/controllers/archived-email.controller.ts +++ b/packages/backend/src/api/controllers/archived-email.controller.ts @@ -3,57 +3,55 @@ import { ArchivedEmailService } from '../../services/ArchivedEmailService'; import { config } from '../../config'; export class ArchivedEmailController { - public getArchivedEmails = async (req: Request, res: Response): Promise => { - try { - const { ingestionSourceId } = req.params; - const page = parseInt(req.query.page as string, 10) || 1; - const limit = parseInt(req.query.limit as string, 10) || 10; + public getArchivedEmails = async (req: Request, res: Response): Promise => { + try { + const { ingestionSourceId } = req.params; + const page = parseInt(req.query.page as string, 10) || 1; + const limit = parseInt(req.query.limit as string, 10) || 10; - const result = await ArchivedEmailService.getArchivedEmails( - ingestionSourceId, - page, - limit - ); - return res.status(200).json(result); - } catch (error) { - console.error('Get archived emails error:', error); - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + const result = await ArchivedEmailService.getArchivedEmails( + ingestionSourceId, + page, + limit + ); + return res.status(200).json(result); + } catch (error) { + console.error('Get archived emails error:', error); + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public getArchivedEmailById = async (req: Request, res: Response): Promise => { - try { - const { id } = req.params; - const email = await ArchivedEmailService.getArchivedEmailById(id); - if (!email) { - return res.status(404).json({ message: 'Archived email not found' }); - } - return res.status(200).json(email); - } catch (error) { - console.error(`Get archived email by id ${req.params.id} error:`, error); - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public getArchivedEmailById = async (req: Request, res: Response): Promise => { + try { + const { id } = req.params; + const email = await ArchivedEmailService.getArchivedEmailById(id); + if (!email) { + return res.status(404).json({ message: 'Archived email not found' }); + } + return res.status(200).json(email); + } catch (error) { + console.error(`Get archived email by id ${req.params.id} error:`, error); + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public deleteArchivedEmail = async (req: Request, res: Response): Promise => { - if (config.app.isDemo) { - return res - .status(403) - .json({ message: 'This operation is not allowed in demo mode.' }); - } - try { - const { id } = req.params; - await ArchivedEmailService.deleteArchivedEmail(id); - return res.status(204).send(); - } catch (error) { - console.error(`Delete archived email ${req.params.id} error:`, error); - if (error instanceof Error) { - if (error.message === 'Archived email not found') { - return res.status(404).json({ message: error.message }); - } - return res.status(500).json({ message: error.message }); - } - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public deleteArchivedEmail = async (req: Request, res: Response): Promise => { + if (config.app.isDemo) { + return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); + } + try { + const { id } = req.params; + await ArchivedEmailService.deleteArchivedEmail(id); + return res.status(204).send(); + } catch (error) { + console.error(`Delete archived email ${req.params.id} error:`, error); + if (error instanceof Error) { + if (error.message === 'Archived email not found') { + return res.status(404).json({ message: error.message }); + } + return res.status(500).json({ message: error.message }); + } + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; } diff --git a/packages/backend/src/api/controllers/auth.controller.ts b/packages/backend/src/api/controllers/auth.controller.ts index 7b760af..23423e5 100644 --- a/packages/backend/src/api/controllers/auth.controller.ts +++ b/packages/backend/src/api/controllers/auth.controller.ts @@ -6,88 +6,94 @@ import * as schema from '../../database/schema'; import { sql } from 'drizzle-orm'; import 'dotenv/config'; - export class AuthController { - #authService: AuthService; - #userService: UserService; + #authService: AuthService; + #userService: UserService; - constructor(authService: AuthService, userService: UserService) { - this.#authService = authService; - this.#userService = userService; - } - /** - * Only used for setting up the instance, should only be displayed once upon instance set up. - * @param req - * @param res - * @returns - */ - public setup = async (req: Request, res: Response): Promise => { - const { email, password, first_name, last_name } = req.body; + constructor(authService: AuthService, userService: UserService) { + this.#authService = authService; + this.#userService = userService; + } + /** + * Only used for setting up the instance, should only be displayed once upon instance set up. + * @param req + * @param res + * @returns + */ + public setup = async (req: Request, res: Response): Promise => { + const { email, password, first_name, last_name } = req.body; - if (!email || !password || !first_name || !last_name) { - return res.status(400).json({ message: 'Email, password, and name are required' }); - } + if (!email || !password || !first_name || !last_name) { + return res.status(400).json({ message: 'Email, password, and name are required' }); + } - try { - const userCountResult = await db.select({ count: sql`count(*)` }).from(schema.users); - const userCount = Number(userCountResult[0].count); + try { + const userCountResult = await db + .select({ count: sql`count(*)` }) + .from(schema.users); + const userCount = Number(userCountResult[0].count); - if (userCount > 0) { - return res.status(403).json({ message: 'Setup has already been completed.' }); - } + if (userCount > 0) { + return res.status(403).json({ message: 'Setup has already been completed.' }); + } - const newUser = await this.#userService.createAdminUser({ email, password, first_name, last_name }, true); - const result = await this.#authService.login(email, password); - return res.status(201).json(result); - } catch (error) { - console.error('Setup error:', error); - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + const newUser = await this.#userService.createAdminUser( + { email, password, first_name, last_name }, + true + ); + const result = await this.#authService.login(email, password); + return res.status(201).json(result); + } catch (error) { + console.error('Setup error:', error); + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public login = async (req: Request, res: Response): Promise => { - const { email, password } = req.body; + public login = async (req: Request, res: Response): Promise => { + const { email, password } = req.body; - if (!email || !password) { - return res.status(400).json({ message: 'Email and password are required' }); - } + if (!email || !password) { + return res.status(400).json({ message: 'Email and password are required' }); + } - try { - const result = await this.#authService.login(email, password); + try { + const result = await this.#authService.login(email, password); - if (!result) { - return res.status(401).json({ message: 'Invalid credentials' }); - } + if (!result) { + return res.status(401).json({ message: 'Invalid credentials' }); + } - return res.status(200).json(result); - } catch (error) { - console.error('Login error:', error); - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + return res.status(200).json(result); + } catch (error) { + console.error('Login error:', error); + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public status = async (req: Request, res: Response): Promise => { - try { - - - - const userCountResult = await db.select({ count: sql`count(*)` }).from(schema.users); - const userCount = Number(userCountResult[0].count); - const needsSetup = userCount === 0; - // in case user uses older version with admin user variables, we will create the admin user using those variables. - if (needsSetup && process.env.ADMIN_EMAIL && process.env.ADMIN_PASSWORD) { - await this.#userService.createAdminUser({ - email: process.env.ADMIN_EMAIL, - password: process.env.ADMIN_PASSWORD, - first_name: "Admin", - last_name: "User" - }, true); - return res.status(200).json({ needsSetup: false }); - } - return res.status(200).json({ needsSetup }); - } catch (error) { - console.error('Status check error:', error); - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public status = async (req: Request, res: Response): Promise => { + try { + const userCountResult = await db + .select({ count: sql`count(*)` }) + .from(schema.users); + const userCount = Number(userCountResult[0].count); + const needsSetup = userCount === 0; + // in case user uses older version with admin user variables, we will create the admin user using those variables. + if (needsSetup && process.env.ADMIN_EMAIL && process.env.ADMIN_PASSWORD) { + await this.#userService.createAdminUser( + { + email: process.env.ADMIN_EMAIL, + password: process.env.ADMIN_PASSWORD, + first_name: 'Admin', + last_name: 'User', + }, + true + ); + return res.status(200).json({ needsSetup: false }); + } + return res.status(200).json({ needsSetup }); + } catch (error) { + console.error('Status check error:', error); + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; } diff --git a/packages/backend/src/api/controllers/dashboard.controller.ts b/packages/backend/src/api/controllers/dashboard.controller.ts index 9b0dabc..05f3123 100644 --- a/packages/backend/src/api/controllers/dashboard.controller.ts +++ b/packages/backend/src/api/controllers/dashboard.controller.ts @@ -2,30 +2,30 @@ import { Request, Response } from 'express'; import { dashboardService } from '../../services/DashboardService'; class DashboardController { - public async getStats(req: Request, res: Response) { - const stats = await dashboardService.getStats(); - res.json(stats); - } + public async getStats(req: Request, res: Response) { + const stats = await dashboardService.getStats(); + res.json(stats); + } - public async getIngestionHistory(req: Request, res: Response) { - const history = await dashboardService.getIngestionHistory(); - res.json(history); - } + public async getIngestionHistory(req: Request, res: Response) { + const history = await dashboardService.getIngestionHistory(); + res.json(history); + } - public async getIngestionSources(req: Request, res: Response) { - const sources = await dashboardService.getIngestionSources(); - res.json(sources); - } + public async getIngestionSources(req: Request, res: Response) { + const sources = await dashboardService.getIngestionSources(); + res.json(sources); + } - public async getRecentSyncs(req: Request, res: Response) { - const syncs = await dashboardService.getRecentSyncs(); - res.json(syncs); - } + public async getRecentSyncs(req: Request, res: Response) { + const syncs = await dashboardService.getRecentSyncs(); + res.json(syncs); + } - public async getIndexedInsights(req: Request, res: Response) { - const insights = await dashboardService.getIndexedInsights(); - res.json(insights); - } + public async getIndexedInsights(req: Request, res: Response) { + const insights = await dashboardService.getIndexedInsights(); + res.json(insights); + } } export const dashboardController = new DashboardController(); diff --git a/packages/backend/src/api/controllers/iam.controller.ts b/packages/backend/src/api/controllers/iam.controller.ts index f24d1f8..77984eb 100644 --- a/packages/backend/src/api/controllers/iam.controller.ts +++ b/packages/backend/src/api/controllers/iam.controller.ts @@ -4,68 +4,68 @@ import { PolicyValidator } from '../../iam-policy/policy-validator'; import type { PolicyStatement } from '@open-archiver/types'; export class IamController { - #iamService: IamService; + #iamService: IamService; - constructor(iamService: IamService) { - this.#iamService = iamService; - } + constructor(iamService: IamService) { + this.#iamService = iamService; + } - public getRoles = async (req: Request, res: Response): Promise => { - try { - const roles = await this.#iamService.getRoles(); - res.status(200).json(roles); - } catch (error) { - res.status(500).json({ error: 'Failed to get roles.' }); - } - }; + public getRoles = async (req: Request, res: Response): Promise => { + try { + const roles = await this.#iamService.getRoles(); + res.status(200).json(roles); + } catch (error) { + res.status(500).json({ error: 'Failed to get roles.' }); + } + }; - public getRoleById = async (req: Request, res: Response): Promise => { - const { id } = req.params; + public getRoleById = async (req: Request, res: Response): Promise => { + const { id } = req.params; - try { - const role = await this.#iamService.getRoleById(id); - if (role) { - res.status(200).json(role); - } else { - res.status(404).json({ error: 'Role not found.' }); - } - } catch (error) { - res.status(500).json({ error: 'Failed to get role.' }); - } - }; + try { + const role = await this.#iamService.getRoleById(id); + if (role) { + res.status(200).json(role); + } else { + res.status(404).json({ error: 'Role not found.' }); + } + } catch (error) { + res.status(500).json({ error: 'Failed to get role.' }); + } + }; - public createRole = async (req: Request, res: Response): Promise => { - const { name, policy } = req.body; + public createRole = async (req: Request, res: Response): Promise => { + const { name, policy } = req.body; - if (!name || !policy) { - res.status(400).json({ error: 'Missing required fields: name and policy.' }); - return; - } + if (!name || !policy) { + res.status(400).json({ error: 'Missing required fields: name and policy.' }); + return; + } - for (const statement of policy) { - const { valid, reason } = PolicyValidator.isValid(statement as PolicyStatement); - if (!valid) { - res.status(400).json({ error: `Invalid policy statement: ${reason}` }); - return; - } - } + for (const statement of policy) { + const { valid, reason } = PolicyValidator.isValid(statement as PolicyStatement); + if (!valid) { + res.status(400).json({ error: `Invalid policy statement: ${reason}` }); + return; + } + } - try { - const role = await this.#iamService.createRole(name, policy); - res.status(201).json(role); - } catch (error) { - res.status(500).json({ error: 'Failed to create role.' }); - } - }; + try { + const role = await this.#iamService.createRole(name, policy); + res.status(201).json(role); + } catch (error) { + res.status(500).json({ error: 'Failed to create role.' }); + } + }; - public deleteRole = async (req: Request, res: Response): Promise => { - const { id } = req.params; + public deleteRole = async (req: Request, res: Response): Promise => { + const { id } = req.params; - try { - await this.#iamService.deleteRole(id); - res.status(204).send(); - } catch (error) { - res.status(500).json({ error: 'Failed to delete role.' }); - } - }; + try { + await this.#iamService.deleteRole(id); + res.status(204).send(); + } catch (error) { + res.status(500).json({ error: 'Failed to delete role.' }); + } + }; } diff --git a/packages/backend/src/api/controllers/ingestion.controller.ts b/packages/backend/src/api/controllers/ingestion.controller.ts index df8ae83..746f1c8 100644 --- a/packages/backend/src/api/controllers/ingestion.controller.ts +++ b/packages/backend/src/api/controllers/ingestion.controller.ts @@ -1,153 +1,159 @@ import { Request, Response } from 'express'; import { IngestionService } from '../../services/IngestionService'; import { - CreateIngestionSourceDto, - UpdateIngestionSourceDto, - IngestionSource, - SafeIngestionSource + CreateIngestionSourceDto, + UpdateIngestionSourceDto, + IngestionSource, + SafeIngestionSource, } from '@open-archiver/types'; import { logger } from '../../config/logger'; import { config } from '../../config'; export class IngestionController { - /** - * Converts an IngestionSource object to a safe version for client-side consumption - * by removing the credentials. - * @param source The full IngestionSource object. - * @returns An object conforming to the SafeIngestionSource type. - */ - private toSafeIngestionSource(source: IngestionSource): SafeIngestionSource { - const { credentials, ...safeSource } = source; - return safeSource; - } + /** + * Converts an IngestionSource object to a safe version for client-side consumption + * by removing the credentials. + * @param source The full IngestionSource object. + * @returns An object conforming to the SafeIngestionSource type. + */ + private toSafeIngestionSource(source: IngestionSource): SafeIngestionSource { + const { credentials, ...safeSource } = source; + return safeSource; + } - public create = async (req: Request, res: Response): Promise => { - if (config.app.isDemo) { - return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); - } - try { - const dto: CreateIngestionSourceDto = req.body; - const newSource = await IngestionService.create(dto); - const safeSource = this.toSafeIngestionSource(newSource); - return res.status(201).json(safeSource); - } catch (error: any) { - logger.error({ err: error }, 'Create ingestion source error'); - // Return a 400 Bad Request for connection errors - return res.status(400).json({ message: error.message || 'Failed to create ingestion source due to a connection error.' }); - } - }; + public create = async (req: Request, res: Response): Promise => { + if (config.app.isDemo) { + return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); + } + try { + const dto: CreateIngestionSourceDto = req.body; + const newSource = await IngestionService.create(dto); + const safeSource = this.toSafeIngestionSource(newSource); + return res.status(201).json(safeSource); + } catch (error: any) { + logger.error({ err: error }, 'Create ingestion source error'); + // Return a 400 Bad Request for connection errors + return res + .status(400) + .json({ + message: + error.message || + 'Failed to create ingestion source due to a connection error.', + }); + } + }; - public findAll = async (req: Request, res: Response): Promise => { - try { - const sources = await IngestionService.findAll(); - const safeSources = sources.map(this.toSafeIngestionSource); - return res.status(200).json(safeSources); - } catch (error) { - console.error('Find all ingestion sources error:', error); - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public findAll = async (req: Request, res: Response): Promise => { + try { + const sources = await IngestionService.findAll(); + const safeSources = sources.map(this.toSafeIngestionSource); + return res.status(200).json(safeSources); + } catch (error) { + console.error('Find all ingestion sources error:', error); + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public findById = async (req: Request, res: Response): Promise => { - try { - const { id } = req.params; - const source = await IngestionService.findById(id); - const safeSource = this.toSafeIngestionSource(source); - return res.status(200).json(safeSource); - } catch (error) { - console.error(`Find ingestion source by id ${req.params.id} error:`, error); - if (error instanceof Error && error.message === 'Ingestion source not found') { - return res.status(404).json({ message: error.message }); - } - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public findById = async (req: Request, res: Response): Promise => { + try { + const { id } = req.params; + const source = await IngestionService.findById(id); + const safeSource = this.toSafeIngestionSource(source); + return res.status(200).json(safeSource); + } catch (error) { + console.error(`Find ingestion source by id ${req.params.id} error:`, error); + if (error instanceof Error && error.message === 'Ingestion source not found') { + return res.status(404).json({ message: error.message }); + } + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public update = async (req: Request, res: Response): Promise => { - if (config.app.isDemo) { - return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); - } - try { - const { id } = req.params; - const dto: UpdateIngestionSourceDto = req.body; - const updatedSource = await IngestionService.update(id, dto); - const safeSource = this.toSafeIngestionSource(updatedSource); - return res.status(200).json(safeSource); - } catch (error) { - console.error(`Update ingestion source ${req.params.id} error:`, error); - if (error instanceof Error && error.message === 'Ingestion source not found') { - return res.status(404).json({ message: error.message }); - } - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public update = async (req: Request, res: Response): Promise => { + if (config.app.isDemo) { + return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); + } + try { + const { id } = req.params; + const dto: UpdateIngestionSourceDto = req.body; + const updatedSource = await IngestionService.update(id, dto); + const safeSource = this.toSafeIngestionSource(updatedSource); + return res.status(200).json(safeSource); + } catch (error) { + console.error(`Update ingestion source ${req.params.id} error:`, error); + if (error instanceof Error && error.message === 'Ingestion source not found') { + return res.status(404).json({ message: error.message }); + } + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public delete = async (req: Request, res: Response): Promise => { - if (config.app.isDemo) { - return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); - } - try { - const { id } = req.params; - await IngestionService.delete(id); - return res.status(204).send(); - } catch (error) { - console.error(`Delete ingestion source ${req.params.id} error:`, error); - if (error instanceof Error && error.message === 'Ingestion source not found') { - return res.status(404).json({ message: error.message }); - } - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public delete = async (req: Request, res: Response): Promise => { + if (config.app.isDemo) { + return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); + } + try { + const { id } = req.params; + await IngestionService.delete(id); + return res.status(204).send(); + } catch (error) { + console.error(`Delete ingestion source ${req.params.id} error:`, error); + if (error instanceof Error && error.message === 'Ingestion source not found') { + return res.status(404).json({ message: error.message }); + } + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public triggerInitialImport = async (req: Request, res: Response): Promise => { - if (config.app.isDemo) { - return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); - } - try { - const { id } = req.params; - await IngestionService.triggerInitialImport(id); - return res.status(202).json({ message: 'Initial import triggered successfully.' }); - } catch (error) { - console.error(`Trigger initial import for ${req.params.id} error:`, error); - if (error instanceof Error && error.message === 'Ingestion source not found') { - return res.status(404).json({ message: error.message }); - } - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public triggerInitialImport = async (req: Request, res: Response): Promise => { + if (config.app.isDemo) { + return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); + } + try { + const { id } = req.params; + await IngestionService.triggerInitialImport(id); + return res.status(202).json({ message: 'Initial import triggered successfully.' }); + } catch (error) { + console.error(`Trigger initial import for ${req.params.id} error:`, error); + if (error instanceof Error && error.message === 'Ingestion source not found') { + return res.status(404).json({ message: error.message }); + } + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public pause = async (req: Request, res: Response): Promise => { - if (config.app.isDemo) { - return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); - } - try { - const { id } = req.params; - const updatedSource = await IngestionService.update(id, { status: 'paused' }); - const safeSource = this.toSafeIngestionSource(updatedSource); - return res.status(200).json(safeSource); - } catch (error) { - console.error(`Pause ingestion source ${req.params.id} error:`, error); - if (error instanceof Error && error.message === 'Ingestion source not found') { - return res.status(404).json({ message: error.message }); - } - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public pause = async (req: Request, res: Response): Promise => { + if (config.app.isDemo) { + return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); + } + try { + const { id } = req.params; + const updatedSource = await IngestionService.update(id, { status: 'paused' }); + const safeSource = this.toSafeIngestionSource(updatedSource); + return res.status(200).json(safeSource); + } catch (error) { + console.error(`Pause ingestion source ${req.params.id} error:`, error); + if (error instanceof Error && error.message === 'Ingestion source not found') { + return res.status(404).json({ message: error.message }); + } + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; - public triggerForceSync = async (req: Request, res: Response): Promise => { - if (config.app.isDemo) { - return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); - } - try { - const { id } = req.params; - await IngestionService.triggerForceSync(id); - return res.status(202).json({ message: 'Force sync triggered successfully.' }); - } catch (error) { - console.error(`Trigger force sync for ${req.params.id} error:`, error); - if (error instanceof Error && error.message === 'Ingestion source not found') { - return res.status(404).json({ message: error.message }); - } - return res.status(500).json({ message: 'An internal server error occurred' }); - } - }; + public triggerForceSync = async (req: Request, res: Response): Promise => { + if (config.app.isDemo) { + return res.status(403).json({ message: 'This operation is not allowed in demo mode.' }); + } + try { + const { id } = req.params; + await IngestionService.triggerForceSync(id); + return res.status(202).json({ message: 'Force sync triggered successfully.' }); + } catch (error) { + console.error(`Trigger force sync for ${req.params.id} error:`, error); + if (error instanceof Error && error.message === 'Ingestion source not found') { + return res.status(404).json({ message: error.message }); + } + return res.status(500).json({ message: 'An internal server error occurred' }); + } + }; } diff --git a/packages/backend/src/api/controllers/search.controller.ts b/packages/backend/src/api/controllers/search.controller.ts index 934c2c6..9f6d4e6 100644 --- a/packages/backend/src/api/controllers/search.controller.ts +++ b/packages/backend/src/api/controllers/search.controller.ts @@ -3,32 +3,32 @@ import { SearchService } from '../../services/SearchService'; import { MatchingStrategies } from 'meilisearch'; export class SearchController { - private searchService: SearchService; + private searchService: SearchService; - constructor() { - this.searchService = new SearchService(); - } + constructor() { + this.searchService = new SearchService(); + } - public search = async (req: Request, res: Response): Promise => { - try { - const { keywords, page, limit, matchingStrategy } = req.query; + public search = async (req: Request, res: Response): Promise => { + try { + const { keywords, page, limit, matchingStrategy } = req.query; - if (!keywords) { - res.status(400).json({ message: 'Keywords are required' }); - return; - } + if (!keywords) { + res.status(400).json({ message: 'Keywords are required' }); + return; + } - const results = await this.searchService.searchEmails({ - query: keywords as string, - page: page ? parseInt(page as string) : 1, - limit: limit ? parseInt(limit as string) : 10, - matchingStrategy: matchingStrategy as MatchingStrategies - }); + const results = await this.searchService.searchEmails({ + query: keywords as string, + page: page ? parseInt(page as string) : 1, + limit: limit ? parseInt(limit as string) : 10, + matchingStrategy: matchingStrategy as MatchingStrategies, + }); - res.status(200).json(results); - } catch (error) { - const message = error instanceof Error ? error.message : 'An unknown error occurred'; - res.status(500).json({ message }); - } - }; + res.status(200).json(results); + } catch (error) { + const message = error instanceof Error ? error.message : 'An unknown error occurred'; + res.status(500).json({ message }); + } + }; } diff --git a/packages/backend/src/api/controllers/storage.controller.ts b/packages/backend/src/api/controllers/storage.controller.ts index eb70da2..f836692 100644 --- a/packages/backend/src/api/controllers/storage.controller.ts +++ b/packages/backend/src/api/controllers/storage.controller.ts @@ -4,47 +4,47 @@ import * as path from 'path'; import { storage as storageConfig } from '../../config/storage'; export class StorageController { - constructor(private storageService: StorageService) { } + constructor(private storageService: StorageService) {} - public downloadFile = async (req: Request, res: Response): Promise => { - const unsafePath = req.query.path as string; + public downloadFile = async (req: Request, res: Response): Promise => { + const unsafePath = req.query.path as string; - if (!unsafePath) { - res.status(400).send('File path is required'); - return; - } + if (!unsafePath) { + res.status(400).send('File path is required'); + return; + } - // Normalize the path to prevent directory traversal - const normalizedPath = path.normalize(unsafePath).replace(/^(\.\.(\/|\\|$))+/, ''); + // Normalize the path to prevent directory traversal + const normalizedPath = path.normalize(unsafePath).replace(/^(\.\.(\/|\\|$))+/, ''); - // Determine the base path from storage configuration - const basePath = storageConfig.type === 'local' ? storageConfig.rootPath : '/'; + // Determine the base path from storage configuration + const basePath = storageConfig.type === 'local' ? storageConfig.rootPath : '/'; - // Resolve the full path and ensure it's within the storage directory - const fullPath = path.join(basePath, normalizedPath); + // Resolve the full path and ensure it's within the storage directory + const fullPath = path.join(basePath, normalizedPath); - if (!fullPath.startsWith(basePath)) { - res.status(400).send('Invalid file path'); - return; - } + if (!fullPath.startsWith(basePath)) { + res.status(400).send('Invalid file path'); + return; + } - // Use the sanitized, relative path for storage service operations - const safePath = path.relative(basePath, fullPath); + // Use the sanitized, relative path for storage service operations + const safePath = path.relative(basePath, fullPath); - try { - const fileExists = await this.storageService.exists(safePath); - if (!fileExists) { - res.status(404).send('File not found'); - return; - } + try { + const fileExists = await this.storageService.exists(safePath); + if (!fileExists) { + res.status(404).send('File not found'); + return; + } - const fileStream = await this.storageService.get(safePath); - const fileName = path.basename(safePath); - res.setHeader('Content-Disposition', `attachment; filename="${fileName}"`); - fileStream.pipe(res); - } catch (error) { - console.error('Error downloading file:', error); - res.status(500).send('Error downloading file'); - } - }; + const fileStream = await this.storageService.get(safePath); + const fileName = path.basename(safePath); + res.setHeader('Content-Disposition', `attachment; filename="${fileName}"`); + fileStream.pipe(res); + } catch (error) { + console.error('Error downloading file:', error); + res.status(500).send('Error downloading file'); + } + }; } diff --git a/packages/backend/src/api/controllers/upload.controller.ts b/packages/backend/src/api/controllers/upload.controller.ts index 9be0144..3d8cad5 100644 --- a/packages/backend/src/api/controllers/upload.controller.ts +++ b/packages/backend/src/api/controllers/upload.controller.ts @@ -4,23 +4,22 @@ import { randomUUID } from 'crypto'; import busboy from 'busboy'; import { config } from '../../config/index'; - export const uploadFile = async (req: Request, res: Response) => { - const storage = new StorageService(); - const bb = busboy({ headers: req.headers }); - let filePath = ''; - let originalFilename = ''; + const storage = new StorageService(); + const bb = busboy({ headers: req.headers }); + let filePath = ''; + let originalFilename = ''; - bb.on('file', (fieldname, file, filename) => { - originalFilename = filename.filename; - const uuid = randomUUID(); - filePath = `${config.storage.openArchiverFolderName}/tmp/${uuid}-${originalFilename}`; - storage.put(filePath, file); - }); + bb.on('file', (fieldname, file, filename) => { + originalFilename = filename.filename; + const uuid = randomUUID(); + filePath = `${config.storage.openArchiverFolderName}/tmp/${uuid}-${originalFilename}`; + storage.put(filePath, file); + }); - bb.on('finish', () => { - res.json({ filePath }); - }); + bb.on('finish', () => { + res.json({ filePath }); + }); - req.pipe(bb); + req.pipe(bb); }; diff --git a/packages/backend/src/api/middleware/rateLimiter.ts b/packages/backend/src/api/middleware/rateLimiter.ts index 7450e93..5e444b6 100644 --- a/packages/backend/src/api/middleware/rateLimiter.ts +++ b/packages/backend/src/api/middleware/rateLimiter.ts @@ -2,9 +2,9 @@ import rateLimit from 'express-rate-limit'; // Rate limiter to prevent brute-force attacks on the login endpoint export const loginRateLimiter = rateLimit({ - windowMs: 15 * 60 * 1000, // 15 minutes - max: 10, // Limit each IP to 10 login requests per windowMs - message: 'Too many login attempts from this IP, please try again after 15 minutes', - standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers - legacyHeaders: false, // Disable the `X-RateLimit-*` headers + windowMs: 15 * 60 * 1000, // 15 minutes + max: 10, // Limit each IP to 10 login requests per windowMs + message: 'Too many login attempts from this IP, please try again after 15 minutes', + standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers + legacyHeaders: false, // Disable the `X-RateLimit-*` headers }); diff --git a/packages/backend/src/api/middleware/requireAuth.ts b/packages/backend/src/api/middleware/requireAuth.ts index 87db157..05e5e38 100644 --- a/packages/backend/src/api/middleware/requireAuth.ts +++ b/packages/backend/src/api/middleware/requireAuth.ts @@ -5,35 +5,37 @@ import 'dotenv/config'; // By using module augmentation, we can add our custom 'user' property // to the Express Request interface in a type-safe way. declare global { - namespace Express { - export interface Request { - user?: AuthTokenPayload; - } - } + namespace Express { + export interface Request { + user?: AuthTokenPayload; + } + } } export const requireAuth = (authService: AuthService) => { - return async (req: Request, res: Response, next: NextFunction) => { - const authHeader = req.headers.authorization; - if (!authHeader || !authHeader.startsWith('Bearer ')) { - return res.status(401).json({ message: 'Unauthorized: No token provided' }); - } - const token = authHeader.split(' ')[1]; - try { - // use a SUPER_API_KEY for all authentications. add process.env.SUPER_API_KEY conditional check in case user didn't set a SUPER_API_KEY. - if (process.env.SUPER_API_KEY && token === process.env.SUPER_API_KEY) { - next(); - return; - } - const payload = await authService.verifyToken(token); - if (!payload) { - return res.status(401).json({ message: 'Unauthorized: Invalid token' }); - } - req.user = payload; - next(); - } catch (error) { - console.error('Authentication error:', error); - return res.status(500).json({ message: 'An internal server error occurred during authentication' }); - } - }; + return async (req: Request, res: Response, next: NextFunction) => { + const authHeader = req.headers.authorization; + if (!authHeader || !authHeader.startsWith('Bearer ')) { + return res.status(401).json({ message: 'Unauthorized: No token provided' }); + } + const token = authHeader.split(' ')[1]; + try { + // use a SUPER_API_KEY for all authentications. add process.env.SUPER_API_KEY conditional check in case user didn't set a SUPER_API_KEY. + if (process.env.SUPER_API_KEY && token === process.env.SUPER_API_KEY) { + next(); + return; + } + const payload = await authService.verifyToken(token); + if (!payload) { + return res.status(401).json({ message: 'Unauthorized: Invalid token' }); + } + req.user = payload; + next(); + } catch (error) { + console.error('Authentication error:', error); + return res + .status(500) + .json({ message: 'An internal server error occurred during authentication' }); + } + }; }; diff --git a/packages/backend/src/api/routes/archived-email.routes.ts b/packages/backend/src/api/routes/archived-email.routes.ts index 954298d..5c0948a 100644 --- a/packages/backend/src/api/routes/archived-email.routes.ts +++ b/packages/backend/src/api/routes/archived-email.routes.ts @@ -4,19 +4,19 @@ import { requireAuth } from '../middleware/requireAuth'; import { AuthService } from '../../services/AuthService'; export const createArchivedEmailRouter = ( - archivedEmailController: ArchivedEmailController, - authService: AuthService + archivedEmailController: ArchivedEmailController, + authService: AuthService ): Router => { - const router = Router(); + const router = Router(); - // Secure all routes in this module - router.use(requireAuth(authService)); + // Secure all routes in this module + router.use(requireAuth(authService)); - router.get('/ingestion-source/:ingestionSourceId', archivedEmailController.getArchivedEmails); + router.get('/ingestion-source/:ingestionSourceId', archivedEmailController.getArchivedEmails); - router.get('/:id', archivedEmailController.getArchivedEmailById); + router.get('/:id', archivedEmailController.getArchivedEmailById); - router.delete('/:id', archivedEmailController.deleteArchivedEmail); + router.delete('/:id', archivedEmailController.deleteArchivedEmail); - return router; + return router; }; diff --git a/packages/backend/src/api/routes/auth.routes.ts b/packages/backend/src/api/routes/auth.routes.ts index 3d42e2e..ea2f295 100644 --- a/packages/backend/src/api/routes/auth.routes.ts +++ b/packages/backend/src/api/routes/auth.routes.ts @@ -3,28 +3,28 @@ import { loginRateLimiter } from '../middleware/rateLimiter'; import type { AuthController } from '../controllers/auth.controller'; export const createAuthRouter = (authController: AuthController): Router => { - const router = Router(); + const router = Router(); - /** - * @route POST /api/v1/auth/setup - * @description Creates the initial administrator user. - * @access Public - */ - router.post('/setup', loginRateLimiter, authController.setup); + /** + * @route POST /api/v1/auth/setup + * @description Creates the initial administrator user. + * @access Public + */ + router.post('/setup', loginRateLimiter, authController.setup); - /** - * @route POST /api/v1/auth/login - * @description Authenticates a user and returns a JWT. - * @access Public - */ - router.post('/login', loginRateLimiter, authController.login); + /** + * @route POST /api/v1/auth/login + * @description Authenticates a user and returns a JWT. + * @access Public + */ + router.post('/login', loginRateLimiter, authController.login); - /** - * @route GET /api/v1/auth/status - * @description Checks if the application has been set up. - * @access Public - */ - router.get('/status', authController.status); + /** + * @route GET /api/v1/auth/status + * @description Checks if the application has been set up. + * @access Public + */ + router.get('/status', authController.status); - return router; + return router; }; diff --git a/packages/backend/src/api/routes/dashboard.routes.ts b/packages/backend/src/api/routes/dashboard.routes.ts index e34d5ea..2c4f5e0 100644 --- a/packages/backend/src/api/routes/dashboard.routes.ts +++ b/packages/backend/src/api/routes/dashboard.routes.ts @@ -4,15 +4,15 @@ import { requireAuth } from '../middleware/requireAuth'; import { AuthService } from '../../services/AuthService'; export const createDashboardRouter = (authService: AuthService): Router => { - const router = Router(); + const router = Router(); - router.use(requireAuth(authService)); + router.use(requireAuth(authService)); - router.get('/stats', dashboardController.getStats); - router.get('/ingestion-history', dashboardController.getIngestionHistory); - router.get('/ingestion-sources', dashboardController.getIngestionSources); - router.get('/recent-syncs', dashboardController.getRecentSyncs); - router.get('/indexed-insights', dashboardController.getIndexedInsights); + router.get('/stats', dashboardController.getStats); + router.get('/ingestion-history', dashboardController.getIngestionHistory); + router.get('/ingestion-sources', dashboardController.getIngestionSources); + router.get('/recent-syncs', dashboardController.getRecentSyncs); + router.get('/indexed-insights', dashboardController.getIndexedInsights); - return router; + return router; }; diff --git a/packages/backend/src/api/routes/iam.routes.ts b/packages/backend/src/api/routes/iam.routes.ts index ad000a7..f427985 100644 --- a/packages/backend/src/api/routes/iam.routes.ts +++ b/packages/backend/src/api/routes/iam.routes.ts @@ -3,34 +3,34 @@ import { requireAuth } from '../middleware/requireAuth'; import type { IamController } from '../controllers/iam.controller'; export const createIamRouter = (iamController: IamController): Router => { - const router = Router(); + const router = Router(); - /** - * @route GET /api/v1/iam/roles - * @description Gets all roles. - * @access Private - */ - router.get('/roles', requireAuth, iamController.getRoles); + /** + * @route GET /api/v1/iam/roles + * @description Gets all roles. + * @access Private + */ + router.get('/roles', requireAuth, iamController.getRoles); - /** - * @route GET /api/v1/iam/roles/:id - * @description Gets a role by ID. - * @access Private - */ - router.get('/roles/:id', requireAuth, iamController.getRoleById); + /** + * @route GET /api/v1/iam/roles/:id + * @description Gets a role by ID. + * @access Private + */ + router.get('/roles/:id', requireAuth, iamController.getRoleById); - /** - * @route POST /api/v1/iam/roles - * @description Creates a new role. - * @access Private - */ - router.post('/roles', requireAuth, iamController.createRole); + /** + * @route POST /api/v1/iam/roles + * @description Creates a new role. + * @access Private + */ + router.post('/roles', requireAuth, iamController.createRole); - /** - * @route DELETE /api/v1/iam/roles/:id - * @description Deletes a role. - * @access Private - */ - router.delete('/roles/:id', requireAuth, iamController.deleteRole); - return router; + /** + * @route DELETE /api/v1/iam/roles/:id + * @description Deletes a role. + * @access Private + */ + router.delete('/roles/:id', requireAuth, iamController.deleteRole); + return router; }; diff --git a/packages/backend/src/api/routes/ingestion.routes.ts b/packages/backend/src/api/routes/ingestion.routes.ts index 92956df..76b7491 100644 --- a/packages/backend/src/api/routes/ingestion.routes.ts +++ b/packages/backend/src/api/routes/ingestion.routes.ts @@ -4,29 +4,29 @@ import { requireAuth } from '../middleware/requireAuth'; import { AuthService } from '../../services/AuthService'; export const createIngestionRouter = ( - ingestionController: IngestionController, - authService: AuthService + ingestionController: IngestionController, + authService: AuthService ): Router => { - const router = Router(); + const router = Router(); - // Secure all routes in this module - router.use(requireAuth(authService)); + // Secure all routes in this module + router.use(requireAuth(authService)); - router.post('/', ingestionController.create); + router.post('/', ingestionController.create); - router.get('/', ingestionController.findAll); + router.get('/', ingestionController.findAll); - router.get('/:id', ingestionController.findById); + router.get('/:id', ingestionController.findById); - router.put('/:id', ingestionController.update); + router.put('/:id', ingestionController.update); - router.delete('/:id', ingestionController.delete); + router.delete('/:id', ingestionController.delete); - router.post('/:id/import', ingestionController.triggerInitialImport); + router.post('/:id/import', ingestionController.triggerInitialImport); - router.post('/:id/pause', ingestionController.pause); + router.post('/:id/pause', ingestionController.pause); - router.post('/:id/sync', ingestionController.triggerForceSync); + router.post('/:id/sync', ingestionController.triggerForceSync); - return router; + return router; }; diff --git a/packages/backend/src/api/routes/search.routes.ts b/packages/backend/src/api/routes/search.routes.ts index 674d29f..9495548 100644 --- a/packages/backend/src/api/routes/search.routes.ts +++ b/packages/backend/src/api/routes/search.routes.ts @@ -4,14 +4,14 @@ import { requireAuth } from '../middleware/requireAuth'; import { AuthService } from '../../services/AuthService'; export const createSearchRouter = ( - searchController: SearchController, - authService: AuthService + searchController: SearchController, + authService: AuthService ): Router => { - const router = Router(); + const router = Router(); - router.use(requireAuth(authService)); + router.use(requireAuth(authService)); - router.get('/', searchController.search); + router.get('/', searchController.search); - return router; + return router; }; diff --git a/packages/backend/src/api/routes/storage.routes.ts b/packages/backend/src/api/routes/storage.routes.ts index f4f24b6..7996743 100644 --- a/packages/backend/src/api/routes/storage.routes.ts +++ b/packages/backend/src/api/routes/storage.routes.ts @@ -4,15 +4,15 @@ import { requireAuth } from '../middleware/requireAuth'; import { AuthService } from '../../services/AuthService'; export const createStorageRouter = ( - storageController: StorageController, - authService: AuthService + storageController: StorageController, + authService: AuthService ): Router => { - const router = Router(); + const router = Router(); - // Secure all routes in this module - router.use(requireAuth(authService)); + // Secure all routes in this module + router.use(requireAuth(authService)); - router.get('/download', storageController.downloadFile); + router.get('/download', storageController.downloadFile); - return router; + return router; }; diff --git a/packages/backend/src/api/routes/test.routes.ts b/packages/backend/src/api/routes/test.routes.ts index 339a72d..0af4873 100644 --- a/packages/backend/src/api/routes/test.routes.ts +++ b/packages/backend/src/api/routes/test.routes.ts @@ -3,7 +3,4 @@ import { ingestionQueue } from '../../jobs/queues'; const router: Router = Router(); - - - export default router; diff --git a/packages/backend/src/api/routes/upload.routes.ts b/packages/backend/src/api/routes/upload.routes.ts index f194c23..e4111d5 100644 --- a/packages/backend/src/api/routes/upload.routes.ts +++ b/packages/backend/src/api/routes/upload.routes.ts @@ -4,11 +4,11 @@ import { requireAuth } from '../middleware/requireAuth'; import { AuthService } from '../../services/AuthService'; export const createUploadRouter = (authService: AuthService): Router => { - const router = Router(); + const router = Router(); - router.use(requireAuth(authService)); + router.use(requireAuth(authService)); - router.post('/', uploadFile); + router.post('/', uploadFile); - return router; + return router; }; diff --git a/packages/backend/src/config/app.ts b/packages/backend/src/config/app.ts index 2a0e26c..f527986 100644 --- a/packages/backend/src/config/app.ts +++ b/packages/backend/src/config/app.ts @@ -1,9 +1,9 @@ import 'dotenv/config'; export const app = { - nodeEnv: process.env.NODE_ENV || 'development', - port: process.env.PORT_BACKEND ? parseInt(process.env.PORT_BACKEND, 10) : 4000, - encryptionKey: process.env.ENCRYPTION_KEY, - isDemo: process.env.IS_DEMO === 'true', - syncFrequency: process.env.SYNC_FREQUENCY || '* * * * *' //default to 1 minute + nodeEnv: process.env.NODE_ENV || 'development', + port: process.env.PORT_BACKEND ? parseInt(process.env.PORT_BACKEND, 10) : 4000, + encryptionKey: process.env.ENCRYPTION_KEY, + isDemo: process.env.IS_DEMO === 'true', + syncFrequency: process.env.SYNC_FREQUENCY || '* * * * *', //default to 1 minute }; diff --git a/packages/backend/src/config/index.ts b/packages/backend/src/config/index.ts index b2f68f2..133f880 100644 --- a/packages/backend/src/config/index.ts +++ b/packages/backend/src/config/index.ts @@ -4,8 +4,8 @@ import { searchConfig } from './search'; import { connection as redisConfig } from './redis'; export const config = { - storage, - app, - search: searchConfig, - redis: redisConfig, + storage, + app, + search: searchConfig, + redis: redisConfig, }; diff --git a/packages/backend/src/config/logger.ts b/packages/backend/src/config/logger.ts index 9b6a76d..98cd83e 100644 --- a/packages/backend/src/config/logger.ts +++ b/packages/backend/src/config/logger.ts @@ -1,11 +1,11 @@ import pino from 'pino'; export const logger = pino({ - level: process.env.LOG_LEVEL || 'info', - transport: { - target: 'pino-pretty', - options: { - colorize: true - } - } + level: process.env.LOG_LEVEL || 'info', + transport: { + target: 'pino-pretty', + options: { + colorize: true, + }, + }, }); diff --git a/packages/backend/src/config/redis.ts b/packages/backend/src/config/redis.ts index e4d1d9e..ec21ca8 100644 --- a/packages/backend/src/config/redis.ts +++ b/packages/backend/src/config/redis.ts @@ -4,16 +4,16 @@ import 'dotenv/config'; * @see https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/guide/connections.md */ const connectionOptions: any = { - host: process.env.REDIS_HOST || 'localhost', - port: (process.env.REDIS_PORT && parseInt(process.env.REDIS_PORT, 10)) || 6379, - password: process.env.REDIS_PASSWORD, - enableReadyCheck: true, + host: process.env.REDIS_HOST || 'localhost', + port: (process.env.REDIS_PORT && parseInt(process.env.REDIS_PORT, 10)) || 6379, + password: process.env.REDIS_PASSWORD, + enableReadyCheck: true, }; if (process.env.REDIS_TLS_ENABLED === 'true') { - connectionOptions.tls = { - rejectUnauthorized: false - }; + connectionOptions.tls = { + rejectUnauthorized: false, + }; } export const connection = connectionOptions; diff --git a/packages/backend/src/config/search.ts b/packages/backend/src/config/search.ts index 851874a..d79632c 100644 --- a/packages/backend/src/config/search.ts +++ b/packages/backend/src/config/search.ts @@ -1,6 +1,6 @@ import 'dotenv/config'; export const searchConfig = { - host: process.env.MEILI_HOST || 'http://127.0.0.1:7700', - apiKey: process.env.MEILI_MASTER_KEY || '', + host: process.env.MEILI_HOST || 'http://127.0.0.1:7700', + apiKey: process.env.MEILI_MASTER_KEY || '', }; diff --git a/packages/backend/src/config/storage.ts b/packages/backend/src/config/storage.ts index 656faef..3d1e4cc 100644 --- a/packages/backend/src/config/storage.ts +++ b/packages/backend/src/config/storage.ts @@ -6,35 +6,35 @@ const openArchiverFolderName = 'open-archiver'; let storageConfig: StorageConfig; if (storageType === 'local') { - if (!process.env.STORAGE_LOCAL_ROOT_PATH) { - throw new Error('STORAGE_LOCAL_ROOT_PATH is not defined in the environment variables'); - } - storageConfig = { - type: 'local', - rootPath: process.env.STORAGE_LOCAL_ROOT_PATH, - openArchiverFolderName: openArchiverFolderName - }; + if (!process.env.STORAGE_LOCAL_ROOT_PATH) { + throw new Error('STORAGE_LOCAL_ROOT_PATH is not defined in the environment variables'); + } + storageConfig = { + type: 'local', + rootPath: process.env.STORAGE_LOCAL_ROOT_PATH, + openArchiverFolderName: openArchiverFolderName, + }; } else if (storageType === 's3') { - if ( - !process.env.STORAGE_S3_ENDPOINT || - !process.env.STORAGE_S3_BUCKET || - !process.env.STORAGE_S3_ACCESS_KEY_ID || - !process.env.STORAGE_S3_SECRET_ACCESS_KEY - ) { - throw new Error('One or more S3 storage environment variables are not defined'); - } - storageConfig = { - type: 's3', - endpoint: process.env.STORAGE_S3_ENDPOINT, - bucket: process.env.STORAGE_S3_BUCKET, - accessKeyId: process.env.STORAGE_S3_ACCESS_KEY_ID, - secretAccessKey: process.env.STORAGE_S3_SECRET_ACCESS_KEY, - region: process.env.STORAGE_S3_REGION, - forcePathStyle: process.env.STORAGE_S3_FORCE_PATH_STYLE === 'true', - openArchiverFolderName: openArchiverFolderName - }; + if ( + !process.env.STORAGE_S3_ENDPOINT || + !process.env.STORAGE_S3_BUCKET || + !process.env.STORAGE_S3_ACCESS_KEY_ID || + !process.env.STORAGE_S3_SECRET_ACCESS_KEY + ) { + throw new Error('One or more S3 storage environment variables are not defined'); + } + storageConfig = { + type: 's3', + endpoint: process.env.STORAGE_S3_ENDPOINT, + bucket: process.env.STORAGE_S3_BUCKET, + accessKeyId: process.env.STORAGE_S3_ACCESS_KEY_ID, + secretAccessKey: process.env.STORAGE_S3_SECRET_ACCESS_KEY, + region: process.env.STORAGE_S3_REGION, + forcePathStyle: process.env.STORAGE_S3_FORCE_PATH_STYLE === 'true', + openArchiverFolderName: openArchiverFolderName, + }; } else { - throw new Error(`Invalid STORAGE_TYPE: ${storageType}`); + throw new Error(`Invalid STORAGE_TYPE: ${storageType}`); } export const storage = storageConfig; diff --git a/packages/backend/src/database/index.ts b/packages/backend/src/database/index.ts index 94744b1..8d6c85a 100644 --- a/packages/backend/src/database/index.ts +++ b/packages/backend/src/database/index.ts @@ -6,7 +6,7 @@ import * as schema from './schema'; import { encodeDatabaseUrl } from '../helpers/db'; if (!process.env.DATABASE_URL) { - throw new Error('DATABASE_URL is not set in the .env file'); + throw new Error('DATABASE_URL is not set in the .env file'); } const connectionString = encodeDatabaseUrl(process.env.DATABASE_URL); diff --git a/packages/backend/src/database/migrate.ts b/packages/backend/src/database/migrate.ts index 610d960..721991c 100644 --- a/packages/backend/src/database/migrate.ts +++ b/packages/backend/src/database/migrate.ts @@ -7,23 +7,23 @@ import { encodeDatabaseUrl } from '../helpers/db'; config(); const runMigrate = async () => { - if (!process.env.DATABASE_URL) { - throw new Error('DATABASE_URL is not set in the .env file'); - } + if (!process.env.DATABASE_URL) { + throw new Error('DATABASE_URL is not set in the .env file'); + } - const connectionString = encodeDatabaseUrl(process.env.DATABASE_URL); - const connection = postgres(connectionString, { max: 1 }); - const db = drizzle(connection); + const connectionString = encodeDatabaseUrl(process.env.DATABASE_URL); + const connection = postgres(connectionString, { max: 1 }); + const db = drizzle(connection); - console.log('Running migrations...'); + console.log('Running migrations...'); - await migrate(db, { migrationsFolder: 'src/database/migrations' }); + await migrate(db, { migrationsFolder: 'src/database/migrations' }); - console.log('Migrations completed!'); - process.exit(0); + console.log('Migrations completed!'); + process.exit(0); }; runMigrate().catch((err) => { - console.error('Migration failed!', err); - process.exit(1); + console.error('Migration failed!', err); + process.exit(1); }); diff --git a/packages/backend/src/database/migrations/meta/0000_snapshot.json b/packages/backend/src/database/migrations/meta/0000_snapshot.json index 596765e..295a10b 100644 --- a/packages/backend/src/database/migrations/meta/0000_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0000_snapshot.json @@ -1,818 +1,770 @@ { - "id": "3fe238cc-60db-4ddb-8945-11db89bdee2b", - "prevId": "00000000-0000-0000-0000-000000000000", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": { - "archived_emails_custodian_id_custodians_id_fk": { - "name": "archived_emails_custodian_id_custodians_id_fk", - "tableFrom": "archived_emails", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "3fe238cc-60db-4ddb-8945-11db89bdee2b", + "prevId": "00000000-0000-0000-0000-000000000000", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "archived_emails_custodian_id_custodians_id_fk": { + "name": "archived_emails_custodian_id_custodians_id_fk", + "tableFrom": "archived_emails", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": ["active", "paused", "error", "pending_auth", "syncing"] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0001_snapshot.json b/packages/backend/src/database/migrations/meta/0001_snapshot.json index 3c015c3..2197e84 100644 --- a/packages/backend/src/database/migrations/meta/0001_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0001_snapshot.json @@ -1,819 +1,770 @@ { - "id": "9f4ccc8d-aafa-43de-abf6-f85034dba904", - "prevId": "3fe238cc-60db-4ddb-8945-11db89bdee2b", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": { - "archived_emails_custodian_id_custodians_id_fk": { - "name": "archived_emails_custodian_id_custodians_id_fk", - "tableFrom": "archived_emails", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "9f4ccc8d-aafa-43de-abf6-f85034dba904", + "prevId": "3fe238cc-60db-4ddb-8945-11db89bdee2b", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "archived_emails_custodian_id_custodians_id_fk": { + "name": "archived_emails_custodian_id_custodians_id_fk", + "tableFrom": "archived_emails", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": ["active", "paused", "error", "pending_auth", "syncing", "auth_success"] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0002_snapshot.json b/packages/backend/src/database/migrations/meta/0002_snapshot.json index 7c5c93b..e1310c6 100644 --- a/packages/backend/src/database/migrations/meta/0002_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0002_snapshot.json @@ -1,819 +1,770 @@ { - "id": "bb68c4a0-16d6-40c6-891d-200348601f91", - "prevId": "9f4ccc8d-aafa-43de-abf6-f85034dba904", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "bb68c4a0-16d6-40c6-891d-200348601f91", + "prevId": "9f4ccc8d-aafa-43de-abf6-f85034dba904", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": ["active", "paused", "error", "pending_auth", "syncing", "auth_success"] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0003_snapshot.json b/packages/backend/src/database/migrations/meta/0003_snapshot.json index e9346ba..ae263b0 100644 --- a/packages/backend/src/database/migrations/meta/0003_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0003_snapshot.json @@ -1,819 +1,770 @@ { - "id": "0a5303e6-3d82-4687-bb45-0267a6c72130", - "prevId": "bb68c4a0-16d6-40c6-891d-200348601f91", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "0a5303e6-3d82-4687-bb45-0267a6c72130", + "prevId": "bb68c4a0-16d6-40c6-891d-200348601f91", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": ["active", "paused", "error", "pending_auth", "syncing", "auth_success"] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0004_snapshot.json b/packages/backend/src/database/migrations/meta/0004_snapshot.json index 634dfcd..437eda0 100644 --- a/packages/backend/src/database/migrations/meta/0004_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0004_snapshot.json @@ -1,820 +1,778 @@ { - "id": "d7e20e44-0813-4b7f-99e7-18563429da6f", - "prevId": "0a5303e6-3d82-4687-bb45-0267a6c72130", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "initial_syncing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "d7e20e44-0813-4b7f-99e7-18563429da6f", + "prevId": "0a5303e6-3d82-4687-bb45-0267a6c72130", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "initial_syncing", + "auth_success" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0005_snapshot.json b/packages/backend/src/database/migrations/meta/0005_snapshot.json index aba9699..13557b2 100644 --- a/packages/backend/src/database/migrations/meta/0005_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0005_snapshot.json @@ -1,820 +1,778 @@ { - "id": "4fa75649-1e65-4c61-8cc5-95add8269925", - "prevId": "d7e20e44-0813-4b7f-99e7-18563429da6f", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "importing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "4fa75649-1e65-4c61-8cc5-95add8269925", + "prevId": "d7e20e44-0813-4b7f-99e7-18563429da6f", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "importing", + "auth_success" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0006_snapshot.json b/packages/backend/src/database/migrations/meta/0006_snapshot.json index 917a769..8a39443 100644 --- a/packages/backend/src/database/migrations/meta/0006_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0006_snapshot.json @@ -1,826 +1,784 @@ { - "id": "bdc9d789-04c7-4d9f-b4ed-00366b0d3603", - "prevId": "4fa75649-1e65-4c61-8cc5-95add8269925", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sync_state": { - "name": "sync_state", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "importing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "bdc9d789-04c7-4d9f-b4ed-00366b0d3603", + "prevId": "4fa75649-1e65-4c61-8cc5-95add8269925", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sync_state": { + "name": "sync_state", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "importing", + "auth_success" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0007_snapshot.json b/packages/backend/src/database/migrations/meta/0007_snapshot.json index e6b58de..0b828ef 100644 --- a/packages/backend/src/database/migrations/meta/0007_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0007_snapshot.json @@ -1,826 +1,784 @@ { - "id": "2a68f80f-b233-43bd-8280-745bee76ca3e", - "prevId": "bdc9d789-04c7-4d9f-b4ed-00366b0d3603", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sync_state": { - "name": "sync_state", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "importing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "2a68f80f-b233-43bd-8280-745bee76ca3e", + "prevId": "bdc9d789-04c7-4d9f-b4ed-00366b0d3603", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sync_state": { + "name": "sync_state", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "importing", + "auth_success" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0009_snapshot.json b/packages/backend/src/database/migrations/meta/0009_snapshot.json index a516965..2a93ef3 100644 --- a/packages/backend/src/database/migrations/meta/0009_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0009_snapshot.json @@ -1,854 +1,812 @@ { - "id": "701eda75-451a-4a6d-87e3-b6658fca65da", - "prevId": "86b6960e-1936-4543-846f-a2d24d6dc5d1", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "thread_id": { - "name": "thread_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "user_email": { - "name": "user_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": { - "thread_id_idx": { - "name": "thread_id_idx", - "columns": [ - { - "expression": "thread_id", - "isExpression": false, - "asc": true, - "nulls": "last" - } - ], - "isUnique": false, - "concurrently": false, - "method": "btree", - "with": {} - } - }, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sync_state": { - "name": "sync_state", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "importing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "701eda75-451a-4a6d-87e3-b6658fca65da", + "prevId": "86b6960e-1936-4543-846f-a2d24d6dc5d1", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "thread_id": { + "name": "thread_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "user_email": { + "name": "user_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "thread_id_idx": { + "name": "thread_id_idx", + "columns": [ + { + "expression": "thread_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sync_state": { + "name": "sync_state", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "importing", + "auth_success" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0010_snapshot.json b/packages/backend/src/database/migrations/meta/0010_snapshot.json index 087f867..ecddb5a 100644 --- a/packages/backend/src/database/migrations/meta/0010_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0010_snapshot.json @@ -1,1087 +1,1026 @@ { - "id": "ab45f75d-f50c-457f-ad40-e62ca27a4e2b", - "prevId": "701eda75-451a-4a6d-87e3-b6658fca65da", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "thread_id": { - "name": "thread_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "user_email": { - "name": "user_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": { - "thread_id_idx": { - "name": "thread_id_idx", - "columns": [ - { - "expression": "thread_id", - "isExpression": false, - "asc": true, - "nulls": "last" - } - ], - "isUnique": false, - "concurrently": false, - "method": "btree", - "with": {} - } - }, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sync_state": { - "name": "sync_state", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.roles": { - "name": "roles", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "policies": { - "name": "policies", - "type": "jsonb", - "primaryKey": false, - "notNull": true, - "default": "'[]'::jsonb" - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "roles_name_unique": { - "name": "roles_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.sessions": { - "name": "sessions", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "text", - "primaryKey": true, - "notNull": true - }, - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "expires_at": { - "name": "expires_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "sessions_user_id_users_id_fk": { - "name": "sessions_user_id_users_id_fk", - "tableFrom": "sessions", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.user_roles": { - "name": "user_roles", - "schema": "", - "columns": { - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "role_id": { - "name": "role_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "user_roles_user_id_users_id_fk": { - "name": "user_roles_user_id_users_id_fk", - "tableFrom": "user_roles", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "user_roles_role_id_roles_id_fk": { - "name": "user_roles_role_id_roles_id_fk", - "tableFrom": "user_roles", - "tableTo": "roles", - "columnsFrom": [ - "role_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "user_roles_user_id_role_id_pk": { - "name": "user_roles_user_id_role_id_pk", - "columns": [ - "user_id", - "role_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.users": { - "name": "users", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "password": { - "name": "password", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "provider": { - "name": "provider", - "type": "text", - "primaryKey": false, - "notNull": false, - "default": "'local'" - }, - "provider_id": { - "name": "provider_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "users_email_unique": { - "name": "users_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "importing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "ab45f75d-f50c-457f-ad40-e62ca27a4e2b", + "prevId": "701eda75-451a-4a6d-87e3-b6658fca65da", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "thread_id": { + "name": "thread_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "user_email": { + "name": "user_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "thread_id_idx": { + "name": "thread_id_idx", + "columns": [ + { + "expression": "thread_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sync_state": { + "name": "sync_state", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.roles": { + "name": "roles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "policies": { + "name": "policies", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "roles_name_unique": { + "name": "roles_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.sessions": { + "name": "sessions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "sessions_user_id_users_id_fk": { + "name": "sessions_user_id_users_id_fk", + "tableFrom": "sessions", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.user_roles": { + "name": "user_roles", + "schema": "", + "columns": { + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "role_id": { + "name": "role_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "user_roles_user_id_users_id_fk": { + "name": "user_roles_user_id_users_id_fk", + "tableFrom": "user_roles", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "user_roles_role_id_roles_id_fk": { + "name": "user_roles_role_id_roles_id_fk", + "tableFrom": "user_roles", + "tableTo": "roles", + "columnsFrom": ["role_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "user_roles_user_id_role_id_pk": { + "name": "user_roles_user_id_role_id_pk", + "columns": ["user_id", "role_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.users": { + "name": "users", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "password": { + "name": "password", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider": { + "name": "provider", + "type": "text", + "primaryKey": false, + "notNull": false, + "default": "'local'" + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "users_email_unique": { + "name": "users_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "importing", + "auth_success" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0011_snapshot.json b/packages/backend/src/database/migrations/meta/0011_snapshot.json index bc8a473..e3fd889 100644 --- a/packages/backend/src/database/migrations/meta/0011_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0011_snapshot.json @@ -1,1093 +1,1032 @@ { - "id": "6252768a-7c7f-4dae-9dbd-d3ea9f647cea", - "prevId": "ab45f75d-f50c-457f-ad40-e62ca27a4e2b", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "thread_id": { - "name": "thread_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "user_email": { - "name": "user_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": { - "thread_id_idx": { - "name": "thread_id_idx", - "columns": [ - { - "expression": "thread_id", - "isExpression": false, - "asc": true, - "nulls": "last" - } - ], - "isUnique": false, - "concurrently": false, - "method": "btree", - "with": {} - } - }, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sync_state": { - "name": "sync_state", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.roles": { - "name": "roles", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "policies": { - "name": "policies", - "type": "jsonb", - "primaryKey": false, - "notNull": true, - "default": "'[]'::jsonb" - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "roles_name_unique": { - "name": "roles_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.sessions": { - "name": "sessions", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "text", - "primaryKey": true, - "notNull": true - }, - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "expires_at": { - "name": "expires_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "sessions_user_id_users_id_fk": { - "name": "sessions_user_id_users_id_fk", - "tableFrom": "sessions", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.user_roles": { - "name": "user_roles", - "schema": "", - "columns": { - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "role_id": { - "name": "role_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "user_roles_user_id_users_id_fk": { - "name": "user_roles_user_id_users_id_fk", - "tableFrom": "user_roles", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "user_roles_role_id_roles_id_fk": { - "name": "user_roles_role_id_roles_id_fk", - "tableFrom": "user_roles", - "tableTo": "roles", - "columnsFrom": [ - "role_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "user_roles_user_id_role_id_pk": { - "name": "user_roles_user_id_role_id_pk", - "columns": [ - "user_id", - "role_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.users": { - "name": "users", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "first_name": { - "name": "first_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "last_name": { - "name": "last_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "password": { - "name": "password", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "provider": { - "name": "provider", - "type": "text", - "primaryKey": false, - "notNull": false, - "default": "'local'" - }, - "provider_id": { - "name": "provider_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "users_email_unique": { - "name": "users_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "importing", - "auth_success" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "6252768a-7c7f-4dae-9dbd-d3ea9f647cea", + "prevId": "ab45f75d-f50c-457f-ad40-e62ca27a4e2b", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "thread_id": { + "name": "thread_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "user_email": { + "name": "user_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "thread_id_idx": { + "name": "thread_id_idx", + "columns": [ + { + "expression": "thread_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sync_state": { + "name": "sync_state", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.roles": { + "name": "roles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "policies": { + "name": "policies", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "roles_name_unique": { + "name": "roles_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.sessions": { + "name": "sessions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "sessions_user_id_users_id_fk": { + "name": "sessions_user_id_users_id_fk", + "tableFrom": "sessions", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.user_roles": { + "name": "user_roles", + "schema": "", + "columns": { + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "role_id": { + "name": "role_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "user_roles_user_id_users_id_fk": { + "name": "user_roles_user_id_users_id_fk", + "tableFrom": "user_roles", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "user_roles_role_id_roles_id_fk": { + "name": "user_roles_role_id_roles_id_fk", + "tableFrom": "user_roles", + "tableTo": "roles", + "columnsFrom": ["role_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "user_roles_user_id_role_id_pk": { + "name": "user_roles_user_id_role_id_pk", + "columns": ["user_id", "role_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.users": { + "name": "users", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "first_name": { + "name": "first_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "last_name": { + "name": "last_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "password": { + "name": "password", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider": { + "name": "provider", + "type": "text", + "primaryKey": false, + "notNull": false, + "default": "'local'" + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "users_email_unique": { + "name": "users_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "importing", + "auth_success" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0012_snapshot.json b/packages/backend/src/database/migrations/meta/0012_snapshot.json index 2980aa9..e360a4a 100644 --- a/packages/backend/src/database/migrations/meta/0012_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0012_snapshot.json @@ -1,1095 +1,1033 @@ { - "id": "02ed9805-d480-483a-b73c-5e03a0e526b7", - "prevId": "6252768a-7c7f-4dae-9dbd-d3ea9f647cea", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "thread_id": { - "name": "thread_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "user_email": { - "name": "user_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": { - "thread_id_idx": { - "name": "thread_id_idx", - "columns": [ - { - "expression": "thread_id", - "isExpression": false, - "asc": true, - "nulls": "last" - } - ], - "isUnique": false, - "concurrently": false, - "method": "btree", - "with": {} - } - }, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sync_state": { - "name": "sync_state", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.roles": { - "name": "roles", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "policies": { - "name": "policies", - "type": "jsonb", - "primaryKey": false, - "notNull": true, - "default": "'[]'::jsonb" - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "roles_name_unique": { - "name": "roles_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.sessions": { - "name": "sessions", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "text", - "primaryKey": true, - "notNull": true - }, - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "expires_at": { - "name": "expires_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "sessions_user_id_users_id_fk": { - "name": "sessions_user_id_users_id_fk", - "tableFrom": "sessions", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.user_roles": { - "name": "user_roles", - "schema": "", - "columns": { - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "role_id": { - "name": "role_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "user_roles_user_id_users_id_fk": { - "name": "user_roles_user_id_users_id_fk", - "tableFrom": "user_roles", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "user_roles_role_id_roles_id_fk": { - "name": "user_roles_role_id_roles_id_fk", - "tableFrom": "user_roles", - "tableTo": "roles", - "columnsFrom": [ - "role_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "user_roles_user_id_role_id_pk": { - "name": "user_roles_user_id_role_id_pk", - "columns": [ - "user_id", - "role_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.users": { - "name": "users", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "first_name": { - "name": "first_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "last_name": { - "name": "last_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "password": { - "name": "password", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "provider": { - "name": "provider", - "type": "text", - "primaryKey": false, - "notNull": false, - "default": "'local'" - }, - "provider_id": { - "name": "provider_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "users_email_unique": { - "name": "users_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap", - "pst_import" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "importing", - "auth_success", - "imported" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "02ed9805-d480-483a-b73c-5e03a0e526b7", + "prevId": "6252768a-7c7f-4dae-9dbd-d3ea9f647cea", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "thread_id": { + "name": "thread_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "user_email": { + "name": "user_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "thread_id_idx": { + "name": "thread_id_idx", + "columns": [ + { + "expression": "thread_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sync_state": { + "name": "sync_state", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.roles": { + "name": "roles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "policies": { + "name": "policies", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "roles_name_unique": { + "name": "roles_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.sessions": { + "name": "sessions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "sessions_user_id_users_id_fk": { + "name": "sessions_user_id_users_id_fk", + "tableFrom": "sessions", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.user_roles": { + "name": "user_roles", + "schema": "", + "columns": { + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "role_id": { + "name": "role_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "user_roles_user_id_users_id_fk": { + "name": "user_roles_user_id_users_id_fk", + "tableFrom": "user_roles", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "user_roles_role_id_roles_id_fk": { + "name": "user_roles_role_id_roles_id_fk", + "tableFrom": "user_roles", + "tableTo": "roles", + "columnsFrom": ["role_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "user_roles_user_id_role_id_pk": { + "name": "user_roles_user_id_role_id_pk", + "columns": ["user_id", "role_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.users": { + "name": "users", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "first_name": { + "name": "first_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "last_name": { + "name": "last_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "password": { + "name": "password", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider": { + "name": "provider", + "type": "text", + "primaryKey": false, + "notNull": false, + "default": "'local'" + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "users_email_unique": { + "name": "users_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap", "pst_import"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "importing", + "auth_success", + "imported" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0013_snapshot.json b/packages/backend/src/database/migrations/meta/0013_snapshot.json index 701cbde..7e46319 100644 --- a/packages/backend/src/database/migrations/meta/0013_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0013_snapshot.json @@ -1,1107 +1,1045 @@ { - "id": "c397c819-e69f-42c7-966c-7b2969741c56", - "prevId": "02ed9805-d480-483a-b73c-5e03a0e526b7", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "thread_id": { - "name": "thread_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "user_email": { - "name": "user_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "path": { - "name": "path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "tags": { - "name": "tags", - "type": "jsonb", - "primaryKey": false, - "notNull": false - } - }, - "indexes": { - "thread_id_idx": { - "name": "thread_id_idx", - "columns": [ - { - "expression": "thread_id", - "isExpression": false, - "asc": true, - "nulls": "last" - } - ], - "isUnique": false, - "concurrently": false, - "method": "btree", - "with": {} - } - }, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sync_state": { - "name": "sync_state", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.roles": { - "name": "roles", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "policies": { - "name": "policies", - "type": "jsonb", - "primaryKey": false, - "notNull": true, - "default": "'[]'::jsonb" - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "roles_name_unique": { - "name": "roles_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.sessions": { - "name": "sessions", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "text", - "primaryKey": true, - "notNull": true - }, - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "expires_at": { - "name": "expires_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "sessions_user_id_users_id_fk": { - "name": "sessions_user_id_users_id_fk", - "tableFrom": "sessions", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.user_roles": { - "name": "user_roles", - "schema": "", - "columns": { - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "role_id": { - "name": "role_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "user_roles_user_id_users_id_fk": { - "name": "user_roles_user_id_users_id_fk", - "tableFrom": "user_roles", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "user_roles_role_id_roles_id_fk": { - "name": "user_roles_role_id_roles_id_fk", - "tableFrom": "user_roles", - "tableTo": "roles", - "columnsFrom": [ - "role_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "user_roles_user_id_role_id_pk": { - "name": "user_roles_user_id_role_id_pk", - "columns": [ - "user_id", - "role_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.users": { - "name": "users", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "first_name": { - "name": "first_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "last_name": { - "name": "last_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "password": { - "name": "password", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "provider": { - "name": "provider", - "type": "text", - "primaryKey": false, - "notNull": false, - "default": "'local'" - }, - "provider_id": { - "name": "provider_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "users_email_unique": { - "name": "users_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap", - "pst_import" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "importing", - "auth_success", - "imported" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "c397c819-e69f-42c7-966c-7b2969741c56", + "prevId": "02ed9805-d480-483a-b73c-5e03a0e526b7", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "thread_id": { + "name": "thread_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "user_email": { + "name": "user_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "path": { + "name": "path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tags": { + "name": "tags", + "type": "jsonb", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "thread_id_idx": { + "name": "thread_id_idx", + "columns": [ + { + "expression": "thread_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sync_state": { + "name": "sync_state", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.roles": { + "name": "roles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "policies": { + "name": "policies", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "roles_name_unique": { + "name": "roles_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.sessions": { + "name": "sessions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "sessions_user_id_users_id_fk": { + "name": "sessions_user_id_users_id_fk", + "tableFrom": "sessions", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.user_roles": { + "name": "user_roles", + "schema": "", + "columns": { + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "role_id": { + "name": "role_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "user_roles_user_id_users_id_fk": { + "name": "user_roles_user_id_users_id_fk", + "tableFrom": "user_roles", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "user_roles_role_id_roles_id_fk": { + "name": "user_roles_role_id_roles_id_fk", + "tableFrom": "user_roles", + "tableTo": "roles", + "columnsFrom": ["role_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "user_roles_user_id_role_id_pk": { + "name": "user_roles_user_id_role_id_pk", + "columns": ["user_id", "role_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.users": { + "name": "users", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "first_name": { + "name": "first_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "last_name": { + "name": "last_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "password": { + "name": "password", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider": { + "name": "provider", + "type": "text", + "primaryKey": false, + "notNull": false, + "default": "'local'" + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "users_email_unique": { + "name": "users_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": ["google_workspace", "microsoft_365", "generic_imap", "pst_import"] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "importing", + "auth_success", + "imported" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/0014_snapshot.json b/packages/backend/src/database/migrations/meta/0014_snapshot.json index 50f5d01..a0733c3 100644 --- a/packages/backend/src/database/migrations/meta/0014_snapshot.json +++ b/packages/backend/src/database/migrations/meta/0014_snapshot.json @@ -1,1108 +1,1051 @@ { - "id": "ad5204da-bb82-4a19-abfa-d30cc284ab27", - "prevId": "c397c819-e69f-42c7-966c-7b2969741c56", - "version": "7", - "dialect": "postgresql", - "tables": { - "public.archived_emails": { - "name": "archived_emails", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "thread_id": { - "name": "thread_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "ingestion_source_id": { - "name": "ingestion_source_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "user_email": { - "name": "user_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "message_id_header": { - "name": "message_id_header", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sent_at": { - "name": "sent_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - }, - "subject": { - "name": "subject", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_name": { - "name": "sender_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sender_email": { - "name": "sender_email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "recipients": { - "name": "recipients", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_hash_sha256": { - "name": "storage_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "is_indexed": { - "name": "is_indexed", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "has_attachments": { - "name": "has_attachments", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "is_on_legal_hold": { - "name": "is_on_legal_hold", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": false - }, - "archived_at": { - "name": "archived_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "path": { - "name": "path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "tags": { - "name": "tags", - "type": "jsonb", - "primaryKey": false, - "notNull": false - } - }, - "indexes": { - "thread_id_idx": { - "name": "thread_id_idx", - "columns": [ - { - "expression": "thread_id", - "isExpression": false, - "asc": true, - "nulls": "last" - } - ], - "isUnique": false, - "concurrently": false, - "method": "btree", - "with": {} - } - }, - "foreignKeys": { - "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { - "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", - "tableFrom": "archived_emails", - "tableTo": "ingestion_sources", - "columnsFrom": [ - "ingestion_source_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.attachments": { - "name": "attachments", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "filename": { - "name": "filename", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "mime_type": { - "name": "mime_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "size_bytes": { - "name": "size_bytes", - "type": "bigint", - "primaryKey": false, - "notNull": true - }, - "content_hash_sha256": { - "name": "content_hash_sha256", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "storage_path": { - "name": "storage_path", - "type": "text", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "attachments_content_hash_sha256_unique": { - "name": "attachments_content_hash_sha256_unique", - "nullsNotDistinct": false, - "columns": [ - "content_hash_sha256" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.email_attachments": { - "name": "email_attachments", - "schema": "", - "columns": { - "email_id": { - "name": "email_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "attachment_id": { - "name": "attachment_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "email_attachments_email_id_archived_emails_id_fk": { - "name": "email_attachments_email_id_archived_emails_id_fk", - "tableFrom": "email_attachments", - "tableTo": "archived_emails", - "columnsFrom": [ - "email_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "email_attachments_attachment_id_attachments_id_fk": { - "name": "email_attachments_attachment_id_attachments_id_fk", - "tableFrom": "email_attachments", - "tableTo": "attachments", - "columnsFrom": [ - "attachment_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "restrict", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "email_attachments_email_id_attachment_id_pk": { - "name": "email_attachments_email_id_attachment_id_pk", - "columns": [ - "email_id", - "attachment_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.audit_logs": { - "name": "audit_logs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "bigserial", - "primaryKey": true, - "notNull": true - }, - "timestamp": { - "name": "timestamp", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "actor_identifier": { - "name": "actor_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "action": { - "name": "action", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "target_type": { - "name": "target_type", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "target_id": { - "name": "target_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "details": { - "name": "details", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "is_tamper_evident": { - "name": "is_tamper_evident", - "type": "boolean", - "primaryKey": false, - "notNull": false, - "default": false - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ediscovery_cases": { - "name": "ediscovery_cases", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'open'" - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "ediscovery_cases_name_unique": { - "name": "ediscovery_cases_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.export_jobs": { - "name": "export_jobs", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "format": { - "name": "format", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "status": { - "name": "status", - "type": "text", - "primaryKey": false, - "notNull": true, - "default": "'pending'" - }, - "query": { - "name": "query", - "type": "jsonb", - "primaryKey": false, - "notNull": true - }, - "file_path": { - "name": "file_path", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_by_identifier": { - "name": "created_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "completed_at": { - "name": "completed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "export_jobs_case_id_ediscovery_cases_id_fk": { - "name": "export_jobs_case_id_ediscovery_cases_id_fk", - "tableFrom": "export_jobs", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "set null", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.legal_holds": { - "name": "legal_holds", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "case_id": { - "name": "case_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "custodian_id": { - "name": "custodian_id", - "type": "uuid", - "primaryKey": false, - "notNull": false - }, - "hold_criteria": { - "name": "hold_criteria", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "reason": { - "name": "reason", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "applied_by_identifier": { - "name": "applied_by_identifier", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "applied_at": { - "name": "applied_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "removed_at": { - "name": "removed_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - } - }, - "indexes": {}, - "foreignKeys": { - "legal_holds_case_id_ediscovery_cases_id_fk": { - "name": "legal_holds_case_id_ediscovery_cases_id_fk", - "tableFrom": "legal_holds", - "tableTo": "ediscovery_cases", - "columnsFrom": [ - "case_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "legal_holds_custodian_id_custodians_id_fk": { - "name": "legal_holds_custodian_id_custodians_id_fk", - "tableFrom": "legal_holds", - "tableTo": "custodians", - "columnsFrom": [ - "custodian_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.retention_policies": { - "name": "retention_policies", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "description": { - "name": "description", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "priority": { - "name": "priority", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "retention_period_days": { - "name": "retention_period_days", - "type": "integer", - "primaryKey": false, - "notNull": true - }, - "action_on_expiry": { - "name": "action_on_expiry", - "type": "retention_action", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "is_enabled": { - "name": "is_enabled", - "type": "boolean", - "primaryKey": false, - "notNull": true, - "default": true - }, - "conditions": { - "name": "conditions", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "retention_policies_name_unique": { - "name": "retention_policies_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.custodians": { - "name": "custodians", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "display_name": { - "name": "display_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "source_type": { - "name": "source_type", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "custodians_email_unique": { - "name": "custodians_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.ingestion_sources": { - "name": "ingestion_sources", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "provider": { - "name": "provider", - "type": "ingestion_provider", - "typeSchema": "public", - "primaryKey": false, - "notNull": true - }, - "credentials": { - "name": "credentials", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "status": { - "name": "status", - "type": "ingestion_status", - "typeSchema": "public", - "primaryKey": false, - "notNull": true, - "default": "'pending_auth'" - }, - "last_sync_started_at": { - "name": "last_sync_started_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_finished_at": { - "name": "last_sync_finished_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": false - }, - "last_sync_status_message": { - "name": "last_sync_status_message", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "sync_state": { - "name": "sync_state", - "type": "jsonb", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.roles": { - "name": "roles", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "name": { - "name": "name", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "policies": { - "name": "policies", - "type": "jsonb", - "primaryKey": false, - "notNull": true, - "default": "'[]'::jsonb" - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "roles_name_unique": { - "name": "roles_name_unique", - "nullsNotDistinct": false, - "columns": [ - "name" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.sessions": { - "name": "sessions", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "text", - "primaryKey": true, - "notNull": true - }, - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "expires_at": { - "name": "expires_at", - "type": "timestamp with time zone", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "sessions_user_id_users_id_fk": { - "name": "sessions_user_id_users_id_fk", - "tableFrom": "sessions", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": {}, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.user_roles": { - "name": "user_roles", - "schema": "", - "columns": { - "user_id": { - "name": "user_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - }, - "role_id": { - "name": "role_id", - "type": "uuid", - "primaryKey": false, - "notNull": true - } - }, - "indexes": {}, - "foreignKeys": { - "user_roles_user_id_users_id_fk": { - "name": "user_roles_user_id_users_id_fk", - "tableFrom": "user_roles", - "tableTo": "users", - "columnsFrom": [ - "user_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - }, - "user_roles_role_id_roles_id_fk": { - "name": "user_roles_role_id_roles_id_fk", - "tableFrom": "user_roles", - "tableTo": "roles", - "columnsFrom": [ - "role_id" - ], - "columnsTo": [ - "id" - ], - "onDelete": "cascade", - "onUpdate": "no action" - } - }, - "compositePrimaryKeys": { - "user_roles_user_id_role_id_pk": { - "name": "user_roles_user_id_role_id_pk", - "columns": [ - "user_id", - "role_id" - ] - } - }, - "uniqueConstraints": {}, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - }, - "public.users": { - "name": "users", - "schema": "", - "columns": { - "id": { - "name": "id", - "type": "uuid", - "primaryKey": true, - "notNull": true, - "default": "gen_random_uuid()" - }, - "email": { - "name": "email", - "type": "text", - "primaryKey": false, - "notNull": true - }, - "first_name": { - "name": "first_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "last_name": { - "name": "last_name", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "password": { - "name": "password", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "provider": { - "name": "provider", - "type": "text", - "primaryKey": false, - "notNull": false, - "default": "'local'" - }, - "provider_id": { - "name": "provider_id", - "type": "text", - "primaryKey": false, - "notNull": false - }, - "created_at": { - "name": "created_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - }, - "updated_at": { - "name": "updated_at", - "type": "timestamp", - "primaryKey": false, - "notNull": true, - "default": "now()" - } - }, - "indexes": {}, - "foreignKeys": {}, - "compositePrimaryKeys": {}, - "uniqueConstraints": { - "users_email_unique": { - "name": "users_email_unique", - "nullsNotDistinct": false, - "columns": [ - "email" - ] - } - }, - "policies": {}, - "checkConstraints": {}, - "isRLSEnabled": false - } - }, - "enums": { - "public.retention_action": { - "name": "retention_action", - "schema": "public", - "values": [ - "delete_permanently", - "notify_admin" - ] - }, - "public.ingestion_provider": { - "name": "ingestion_provider", - "schema": "public", - "values": [ - "google_workspace", - "microsoft_365", - "generic_imap", - "pst_import", - "eml_import" - ] - }, - "public.ingestion_status": { - "name": "ingestion_status", - "schema": "public", - "values": [ - "active", - "paused", - "error", - "pending_auth", - "syncing", - "importing", - "auth_success", - "imported" - ] - } - }, - "schemas": {}, - "sequences": {}, - "roles": {}, - "policies": {}, - "views": {}, - "_meta": { - "columns": {}, - "schemas": {}, - "tables": {} - } -} \ No newline at end of file + "id": "ad5204da-bb82-4a19-abfa-d30cc284ab27", + "prevId": "c397c819-e69f-42c7-966c-7b2969741c56", + "version": "7", + "dialect": "postgresql", + "tables": { + "public.archived_emails": { + "name": "archived_emails", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "thread_id": { + "name": "thread_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "ingestion_source_id": { + "name": "ingestion_source_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "user_email": { + "name": "user_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "message_id_header": { + "name": "message_id_header", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sent_at": { + "name": "sent_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "subject": { + "name": "subject", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_name": { + "name": "sender_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sender_email": { + "name": "sender_email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "recipients": { + "name": "recipients", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_hash_sha256": { + "name": "storage_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "is_indexed": { + "name": "is_indexed", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "has_attachments": { + "name": "has_attachments", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_on_legal_hold": { + "name": "is_on_legal_hold", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "archived_at": { + "name": "archived_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "path": { + "name": "path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "tags": { + "name": "tags", + "type": "jsonb", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "thread_id_idx": { + "name": "thread_id_idx", + "columns": [ + { + "expression": "thread_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "archived_emails_ingestion_source_id_ingestion_sources_id_fk": { + "name": "archived_emails_ingestion_source_id_ingestion_sources_id_fk", + "tableFrom": "archived_emails", + "tableTo": "ingestion_sources", + "columnsFrom": ["ingestion_source_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.attachments": { + "name": "attachments", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "filename": { + "name": "filename", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "mime_type": { + "name": "mime_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "size_bytes": { + "name": "size_bytes", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "content_hash_sha256": { + "name": "content_hash_sha256", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "storage_path": { + "name": "storage_path", + "type": "text", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "attachments_content_hash_sha256_unique": { + "name": "attachments_content_hash_sha256_unique", + "nullsNotDistinct": false, + "columns": ["content_hash_sha256"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.email_attachments": { + "name": "email_attachments", + "schema": "", + "columns": { + "email_id": { + "name": "email_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "attachment_id": { + "name": "attachment_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "email_attachments_email_id_archived_emails_id_fk": { + "name": "email_attachments_email_id_archived_emails_id_fk", + "tableFrom": "email_attachments", + "tableTo": "archived_emails", + "columnsFrom": ["email_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "email_attachments_attachment_id_attachments_id_fk": { + "name": "email_attachments_attachment_id_attachments_id_fk", + "tableFrom": "email_attachments", + "tableTo": "attachments", + "columnsFrom": ["attachment_id"], + "columnsTo": ["id"], + "onDelete": "restrict", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "email_attachments_email_id_attachment_id_pk": { + "name": "email_attachments_email_id_attachment_id_pk", + "columns": ["email_id", "attachment_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.audit_logs": { + "name": "audit_logs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "bigserial", + "primaryKey": true, + "notNull": true + }, + "timestamp": { + "name": "timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "actor_identifier": { + "name": "actor_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "action": { + "name": "action", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "target_type": { + "name": "target_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "target_id": { + "name": "target_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "details": { + "name": "details", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "is_tamper_evident": { + "name": "is_tamper_evident", + "type": "boolean", + "primaryKey": false, + "notNull": false, + "default": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ediscovery_cases": { + "name": "ediscovery_cases", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'open'" + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "ediscovery_cases_name_unique": { + "name": "ediscovery_cases_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.export_jobs": { + "name": "export_jobs", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "format": { + "name": "format", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "query": { + "name": "query", + "type": "jsonb", + "primaryKey": false, + "notNull": true + }, + "file_path": { + "name": "file_path", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_by_identifier": { + "name": "created_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "completed_at": { + "name": "completed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "export_jobs_case_id_ediscovery_cases_id_fk": { + "name": "export_jobs_case_id_ediscovery_cases_id_fk", + "tableFrom": "export_jobs", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.legal_holds": { + "name": "legal_holds", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "case_id": { + "name": "case_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "custodian_id": { + "name": "custodian_id", + "type": "uuid", + "primaryKey": false, + "notNull": false + }, + "hold_criteria": { + "name": "hold_criteria", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "reason": { + "name": "reason", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "applied_by_identifier": { + "name": "applied_by_identifier", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "applied_at": { + "name": "applied_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "removed_at": { + "name": "removed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "legal_holds_case_id_ediscovery_cases_id_fk": { + "name": "legal_holds_case_id_ediscovery_cases_id_fk", + "tableFrom": "legal_holds", + "tableTo": "ediscovery_cases", + "columnsFrom": ["case_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "legal_holds_custodian_id_custodians_id_fk": { + "name": "legal_holds_custodian_id_custodians_id_fk", + "tableFrom": "legal_holds", + "tableTo": "custodians", + "columnsFrom": ["custodian_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.retention_policies": { + "name": "retention_policies", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "priority": { + "name": "priority", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "retention_period_days": { + "name": "retention_period_days", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "action_on_expiry": { + "name": "action_on_expiry", + "type": "retention_action", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "is_enabled": { + "name": "is_enabled", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "conditions": { + "name": "conditions", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "retention_policies_name_unique": { + "name": "retention_policies_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.custodians": { + "name": "custodians", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "display_name": { + "name": "display_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_type": { + "name": "source_type", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "custodians_email_unique": { + "name": "custodians_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.ingestion_sources": { + "name": "ingestion_sources", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "provider": { + "name": "provider", + "type": "ingestion_provider", + "typeSchema": "public", + "primaryKey": false, + "notNull": true + }, + "credentials": { + "name": "credentials", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "status": { + "name": "status", + "type": "ingestion_status", + "typeSchema": "public", + "primaryKey": false, + "notNull": true, + "default": "'pending_auth'" + }, + "last_sync_started_at": { + "name": "last_sync_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_finished_at": { + "name": "last_sync_finished_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_sync_status_message": { + "name": "last_sync_status_message", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "sync_state": { + "name": "sync_state", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.roles": { + "name": "roles", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "policies": { + "name": "policies", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'[]'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "roles_name_unique": { + "name": "roles_name_unique", + "nullsNotDistinct": false, + "columns": ["name"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.sessions": { + "name": "sessions", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "expires_at": { + "name": "expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "sessions_user_id_users_id_fk": { + "name": "sessions_user_id_users_id_fk", + "tableFrom": "sessions", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.user_roles": { + "name": "user_roles", + "schema": "", + "columns": { + "user_id": { + "name": "user_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + }, + "role_id": { + "name": "role_id", + "type": "uuid", + "primaryKey": false, + "notNull": true + } + }, + "indexes": {}, + "foreignKeys": { + "user_roles_user_id_users_id_fk": { + "name": "user_roles_user_id_users_id_fk", + "tableFrom": "user_roles", + "tableTo": "users", + "columnsFrom": ["user_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "user_roles_role_id_roles_id_fk": { + "name": "user_roles_role_id_roles_id_fk", + "tableFrom": "user_roles", + "tableTo": "roles", + "columnsFrom": ["role_id"], + "columnsTo": ["id"], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "user_roles_user_id_role_id_pk": { + "name": "user_roles_user_id_role_id_pk", + "columns": ["user_id", "role_id"] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "public.users": { + "name": "users", + "schema": "", + "columns": { + "id": { + "name": "id", + "type": "uuid", + "primaryKey": true, + "notNull": true, + "default": "gen_random_uuid()" + }, + "email": { + "name": "email", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "first_name": { + "name": "first_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "last_name": { + "name": "last_name", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "password": { + "name": "password", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "provider": { + "name": "provider", + "type": "text", + "primaryKey": false, + "notNull": false, + "default": "'local'" + }, + "provider_id": { + "name": "provider_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "users_email_unique": { + "name": "users_email_unique", + "nullsNotDistinct": false, + "columns": ["email"] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + } + }, + "enums": { + "public.retention_action": { + "name": "retention_action", + "schema": "public", + "values": ["delete_permanently", "notify_admin"] + }, + "public.ingestion_provider": { + "name": "ingestion_provider", + "schema": "public", + "values": [ + "google_workspace", + "microsoft_365", + "generic_imap", + "pst_import", + "eml_import" + ] + }, + "public.ingestion_status": { + "name": "ingestion_status", + "schema": "public", + "values": [ + "active", + "paused", + "error", + "pending_auth", + "syncing", + "importing", + "auth_success", + "imported" + ] + } + }, + "schemas": {}, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} diff --git a/packages/backend/src/database/migrations/meta/_journal.json b/packages/backend/src/database/migrations/meta/_journal.json index a969db7..b0d891e 100644 --- a/packages/backend/src/database/migrations/meta/_journal.json +++ b/packages/backend/src/database/migrations/meta/_journal.json @@ -1,111 +1,111 @@ { - "version": "7", - "dialect": "postgresql", - "entries": [ - { - "idx": 0, - "version": "7", - "when": 1752225352591, - "tag": "0000_amusing_namora", - "breakpoints": true - }, - { - "idx": 1, - "version": "7", - "when": 1752326803882, - "tag": "0001_odd_night_thrasher", - "breakpoints": true - }, - { - "idx": 2, - "version": "7", - "when": 1752332648392, - "tag": "0002_lethal_quentin_quire", - "breakpoints": true - }, - { - "idx": 3, - "version": "7", - "when": 1752332967084, - "tag": "0003_petite_wrecker", - "breakpoints": true - }, - { - "idx": 4, - "version": "7", - "when": 1752606108876, - "tag": "0004_sleepy_paper_doll", - "breakpoints": true - }, - { - "idx": 5, - "version": "7", - "when": 1752606327253, - "tag": "0005_chunky_sue_storm", - "breakpoints": true - }, - { - "idx": 6, - "version": "7", - "when": 1753112018514, - "tag": "0006_majestic_caretaker", - "breakpoints": true - }, - { - "idx": 7, - "version": "7", - "when": 1753190159356, - "tag": "0007_handy_archangel", - "breakpoints": true - }, - { - "idx": 8, - "version": "7", - "when": 1753370737317, - "tag": "0008_eminent_the_spike", - "breakpoints": true - }, - { - "idx": 9, - "version": "7", - "when": 1754337938241, - "tag": "0009_late_lenny_balinger", - "breakpoints": true - }, - { - "idx": 10, - "version": "7", - "when": 1754420780849, - "tag": "0010_perpetual_lightspeed", - "breakpoints": true - }, - { - "idx": 11, - "version": "7", - "when": 1754422064158, - "tag": "0011_tan_blackheart", - "breakpoints": true - }, - { - "idx": 12, - "version": "7", - "when": 1754476962901, - "tag": "0012_warm_the_stranger", - "breakpoints": true - }, - { - "idx": 13, - "version": "7", - "when": 1754659373517, - "tag": "0013_classy_talkback", - "breakpoints": true - }, - { - "idx": 14, - "version": "7", - "when": 1754831765718, - "tag": "0014_foamy_vapor", - "breakpoints": true - } - ] -} \ No newline at end of file + "version": "7", + "dialect": "postgresql", + "entries": [ + { + "idx": 0, + "version": "7", + "when": 1752225352591, + "tag": "0000_amusing_namora", + "breakpoints": true + }, + { + "idx": 1, + "version": "7", + "when": 1752326803882, + "tag": "0001_odd_night_thrasher", + "breakpoints": true + }, + { + "idx": 2, + "version": "7", + "when": 1752332648392, + "tag": "0002_lethal_quentin_quire", + "breakpoints": true + }, + { + "idx": 3, + "version": "7", + "when": 1752332967084, + "tag": "0003_petite_wrecker", + "breakpoints": true + }, + { + "idx": 4, + "version": "7", + "when": 1752606108876, + "tag": "0004_sleepy_paper_doll", + "breakpoints": true + }, + { + "idx": 5, + "version": "7", + "when": 1752606327253, + "tag": "0005_chunky_sue_storm", + "breakpoints": true + }, + { + "idx": 6, + "version": "7", + "when": 1753112018514, + "tag": "0006_majestic_caretaker", + "breakpoints": true + }, + { + "idx": 7, + "version": "7", + "when": 1753190159356, + "tag": "0007_handy_archangel", + "breakpoints": true + }, + { + "idx": 8, + "version": "7", + "when": 1753370737317, + "tag": "0008_eminent_the_spike", + "breakpoints": true + }, + { + "idx": 9, + "version": "7", + "when": 1754337938241, + "tag": "0009_late_lenny_balinger", + "breakpoints": true + }, + { + "idx": 10, + "version": "7", + "when": 1754420780849, + "tag": "0010_perpetual_lightspeed", + "breakpoints": true + }, + { + "idx": 11, + "version": "7", + "when": 1754422064158, + "tag": "0011_tan_blackheart", + "breakpoints": true + }, + { + "idx": 12, + "version": "7", + "when": 1754476962901, + "tag": "0012_warm_the_stranger", + "breakpoints": true + }, + { + "idx": 13, + "version": "7", + "when": 1754659373517, + "tag": "0013_classy_talkback", + "breakpoints": true + }, + { + "idx": 14, + "version": "7", + "when": 1754831765718, + "tag": "0014_foamy_vapor", + "breakpoints": true + } + ] +} diff --git a/packages/backend/src/database/schema/archived-emails.ts b/packages/backend/src/database/schema/archived-emails.ts index 750848a..ef90f46 100644 --- a/packages/backend/src/database/schema/archived-emails.ts +++ b/packages/backend/src/database/schema/archived-emails.ts @@ -3,36 +3,36 @@ import { boolean, jsonb, pgTable, text, timestamp, uuid, bigint, index } from 'd import { ingestionSources } from './ingestion-sources'; export const archivedEmails = pgTable( - 'archived_emails', - { - id: uuid('id').primaryKey().defaultRandom(), - threadId: text('thread_id'), - ingestionSourceId: uuid('ingestion_source_id') - .notNull() - .references(() => ingestionSources.id, { onDelete: 'cascade' }), - userEmail: text('user_email').notNull(), - messageIdHeader: text('message_id_header'), - sentAt: timestamp('sent_at', { withTimezone: true }).notNull(), - subject: text('subject'), - senderName: text('sender_name'), - senderEmail: text('sender_email').notNull(), - recipients: jsonb('recipients'), - storagePath: text('storage_path').notNull(), - storageHashSha256: text('storage_hash_sha256').notNull(), - sizeBytes: bigint('size_bytes', { mode: 'number' }).notNull(), - isIndexed: boolean('is_indexed').notNull().default(false), - hasAttachments: boolean('has_attachments').notNull().default(false), - isOnLegalHold: boolean('is_on_legal_hold').notNull().default(false), - archivedAt: timestamp('archived_at', { withTimezone: true }).notNull().defaultNow(), - path: text('path'), - tags: jsonb('tags'), - }, - (table) => [index('thread_id_idx').on(table.threadId)] + 'archived_emails', + { + id: uuid('id').primaryKey().defaultRandom(), + threadId: text('thread_id'), + ingestionSourceId: uuid('ingestion_source_id') + .notNull() + .references(() => ingestionSources.id, { onDelete: 'cascade' }), + userEmail: text('user_email').notNull(), + messageIdHeader: text('message_id_header'), + sentAt: timestamp('sent_at', { withTimezone: true }).notNull(), + subject: text('subject'), + senderName: text('sender_name'), + senderEmail: text('sender_email').notNull(), + recipients: jsonb('recipients'), + storagePath: text('storage_path').notNull(), + storageHashSha256: text('storage_hash_sha256').notNull(), + sizeBytes: bigint('size_bytes', { mode: 'number' }).notNull(), + isIndexed: boolean('is_indexed').notNull().default(false), + hasAttachments: boolean('has_attachments').notNull().default(false), + isOnLegalHold: boolean('is_on_legal_hold').notNull().default(false), + archivedAt: timestamp('archived_at', { withTimezone: true }).notNull().defaultNow(), + path: text('path'), + tags: jsonb('tags'), + }, + (table) => [index('thread_id_idx').on(table.threadId)] ); export const archivedEmailsRelations = relations(archivedEmails, ({ one }) => ({ - ingestionSource: one(ingestionSources, { - fields: [archivedEmails.ingestionSourceId], - references: [ingestionSources.id] - }) + ingestionSource: one(ingestionSources, { + fields: [archivedEmails.ingestionSourceId], + references: [ingestionSources.id], + }), })); diff --git a/packages/backend/src/database/schema/attachments.ts b/packages/backend/src/database/schema/attachments.ts index 35e4bea..c15d4c2 100644 --- a/packages/backend/src/database/schema/attachments.ts +++ b/packages/backend/src/database/schema/attachments.ts @@ -3,32 +3,40 @@ import { pgTable, text, uuid, bigint, primaryKey } from 'drizzle-orm/pg-core'; import { archivedEmails } from './archived-emails'; export const attachments = pgTable('attachments', { - id: uuid('id').primaryKey().defaultRandom(), - filename: text('filename').notNull(), - mimeType: text('mime_type'), - sizeBytes: bigint('size_bytes', { mode: 'number' }).notNull(), - contentHashSha256: text('content_hash_sha256').notNull().unique(), - storagePath: text('storage_path').notNull(), + id: uuid('id').primaryKey().defaultRandom(), + filename: text('filename').notNull(), + mimeType: text('mime_type'), + sizeBytes: bigint('size_bytes', { mode: 'number' }).notNull(), + contentHashSha256: text('content_hash_sha256').notNull().unique(), + storagePath: text('storage_path').notNull(), }); -export const emailAttachments = pgTable('email_attachments', { - emailId: uuid('email_id').notNull().references(() => archivedEmails.id, { onDelete: 'cascade' }), - attachmentId: uuid('attachment_id').notNull().references(() => attachments.id, { onDelete: 'restrict' }), -}, (t) => ({ - pk: primaryKey({ columns: [t.emailId, t.attachmentId] }), -})); +export const emailAttachments = pgTable( + 'email_attachments', + { + emailId: uuid('email_id') + .notNull() + .references(() => archivedEmails.id, { onDelete: 'cascade' }), + attachmentId: uuid('attachment_id') + .notNull() + .references(() => attachments.id, { onDelete: 'restrict' }), + }, + (t) => ({ + pk: primaryKey({ columns: [t.emailId, t.attachmentId] }), + }) +); export const attachmentsRelations = relations(attachments, ({ many }) => ({ - emailAttachments: many(emailAttachments), + emailAttachments: many(emailAttachments), })); export const emailAttachmentsRelations = relations(emailAttachments, ({ one }) => ({ - archivedEmail: one(archivedEmails, { - fields: [emailAttachments.emailId], - references: [archivedEmails.id], - }), - attachment: one(attachments, { - fields: [emailAttachments.attachmentId], - references: [attachments.id], - }), + archivedEmail: one(archivedEmails, { + fields: [emailAttachments.emailId], + references: [archivedEmails.id], + }), + attachment: one(attachments, { + fields: [emailAttachments.attachmentId], + references: [attachments.id], + }), })); diff --git a/packages/backend/src/database/schema/audit-logs.ts b/packages/backend/src/database/schema/audit-logs.ts index 8edba8a..ec667ba 100644 --- a/packages/backend/src/database/schema/audit-logs.ts +++ b/packages/backend/src/database/schema/audit-logs.ts @@ -1,12 +1,12 @@ import { bigserial, boolean, jsonb, pgTable, text, timestamp } from 'drizzle-orm/pg-core'; export const auditLogs = pgTable('audit_logs', { - id: bigserial('id', { mode: 'number' }).primaryKey(), - timestamp: timestamp('timestamp', { withTimezone: true }).notNull().defaultNow(), - actorIdentifier: text('actor_identifier').notNull(), - action: text('action').notNull(), - targetType: text('target_type'), - targetId: text('target_id'), - details: jsonb('details'), - isTamperEvident: boolean('is_tamper_evident').default(false), + id: bigserial('id', { mode: 'number' }).primaryKey(), + timestamp: timestamp('timestamp', { withTimezone: true }).notNull().defaultNow(), + actorIdentifier: text('actor_identifier').notNull(), + action: text('action').notNull(), + targetType: text('target_type'), + targetId: text('target_id'), + details: jsonb('details'), + isTamperEvident: boolean('is_tamper_evident').default(false), }); diff --git a/packages/backend/src/database/schema/compliance.ts b/packages/backend/src/database/schema/compliance.ts index 38369fa..dca199c 100644 --- a/packages/backend/src/database/schema/compliance.ts +++ b/packages/backend/src/database/schema/compliance.ts @@ -1,80 +1,94 @@ import { relations } from 'drizzle-orm'; -import { boolean, integer, jsonb, pgEnum, pgTable, text, timestamp, uuid } from 'drizzle-orm/pg-core'; +import { + boolean, + integer, + jsonb, + pgEnum, + pgTable, + text, + timestamp, + uuid, +} from 'drizzle-orm/pg-core'; import { custodians } from './custodians'; // --- Enums --- -export const retentionActionEnum = pgEnum('retention_action', ['delete_permanently', 'notify_admin']); +export const retentionActionEnum = pgEnum('retention_action', [ + 'delete_permanently', + 'notify_admin', +]); // --- Tables --- export const retentionPolicies = pgTable('retention_policies', { - id: uuid('id').primaryKey().defaultRandom(), - name: text('name').notNull().unique(), - description: text('description'), - priority: integer('priority').notNull(), - retentionPeriodDays: integer('retention_period_days').notNull(), - actionOnExpiry: retentionActionEnum('action_on_expiry').notNull(), - isEnabled: boolean('is_enabled').notNull().default(true), - conditions: jsonb('conditions'), - createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), - updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(), + id: uuid('id').primaryKey().defaultRandom(), + name: text('name').notNull().unique(), + description: text('description'), + priority: integer('priority').notNull(), + retentionPeriodDays: integer('retention_period_days').notNull(), + actionOnExpiry: retentionActionEnum('action_on_expiry').notNull(), + isEnabled: boolean('is_enabled').notNull().default(true), + conditions: jsonb('conditions'), + createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), + updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(), }); export const ediscoveryCases = pgTable('ediscovery_cases', { - id: uuid('id').primaryKey().defaultRandom(), - name: text('name').notNull().unique(), - description: text('description'), - status: text('status').notNull().default('open'), - createdByIdentifier: text('created_by_identifier').notNull(), - createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), - updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(), + id: uuid('id').primaryKey().defaultRandom(), + name: text('name').notNull().unique(), + description: text('description'), + status: text('status').notNull().default('open'), + createdByIdentifier: text('created_by_identifier').notNull(), + createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), + updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(), }); export const legalHolds = pgTable('legal_holds', { - id: uuid('id').primaryKey().defaultRandom(), - caseId: uuid('case_id').notNull().references(() => ediscoveryCases.id, { onDelete: 'cascade' }), - custodianId: uuid('custodian_id').references(() => custodians.id, { onDelete: 'cascade' }), - holdCriteria: jsonb('hold_criteria'), - reason: text('reason'), - appliedByIdentifier: text('applied_by_identifier').notNull(), - appliedAt: timestamp('applied_at', { withTimezone: true }).notNull().defaultNow(), - removedAt: timestamp('removed_at', { withTimezone: true }), + id: uuid('id').primaryKey().defaultRandom(), + caseId: uuid('case_id') + .notNull() + .references(() => ediscoveryCases.id, { onDelete: 'cascade' }), + custodianId: uuid('custodian_id').references(() => custodians.id, { onDelete: 'cascade' }), + holdCriteria: jsonb('hold_criteria'), + reason: text('reason'), + appliedByIdentifier: text('applied_by_identifier').notNull(), + appliedAt: timestamp('applied_at', { withTimezone: true }).notNull().defaultNow(), + removedAt: timestamp('removed_at', { withTimezone: true }), }); export const exportJobs = pgTable('export_jobs', { - id: uuid('id').primaryKey().defaultRandom(), - caseId: uuid('case_id').references(() => ediscoveryCases.id, { onDelete: 'set null' }), - format: text('format').notNull(), - status: text('status').notNull().default('pending'), - query: jsonb('query').notNull(), - filePath: text('file_path'), - createdByIdentifier: text('created_by_identifier').notNull(), - createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), - completedAt: timestamp('completed_at', { withTimezone: true }), + id: uuid('id').primaryKey().defaultRandom(), + caseId: uuid('case_id').references(() => ediscoveryCases.id, { onDelete: 'set null' }), + format: text('format').notNull(), + status: text('status').notNull().default('pending'), + query: jsonb('query').notNull(), + filePath: text('file_path'), + createdByIdentifier: text('created_by_identifier').notNull(), + createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), + completedAt: timestamp('completed_at', { withTimezone: true }), }); // --- Relations --- export const ediscoveryCasesRelations = relations(ediscoveryCases, ({ many }) => ({ - legalHolds: many(legalHolds), - exportJobs: many(exportJobs), + legalHolds: many(legalHolds), + exportJobs: many(exportJobs), })); export const legalHoldsRelations = relations(legalHolds, ({ one }) => ({ - ediscoveryCase: one(ediscoveryCases, { - fields: [legalHolds.caseId], - references: [ediscoveryCases.id], - }), - custodian: one(custodians, { - fields: [legalHolds.custodianId], - references: [custodians.id], - }), + ediscoveryCase: one(ediscoveryCases, { + fields: [legalHolds.caseId], + references: [ediscoveryCases.id], + }), + custodian: one(custodians, { + fields: [legalHolds.custodianId], + references: [custodians.id], + }), })); export const exportJobsRelations = relations(exportJobs, ({ one }) => ({ - ediscoveryCase: one(ediscoveryCases, { - fields: [exportJobs.caseId], - references: [ediscoveryCases.id], - }), + ediscoveryCase: one(ediscoveryCases, { + fields: [exportJobs.caseId], + references: [ediscoveryCases.id], + }), })); diff --git a/packages/backend/src/database/schema/custodians.ts b/packages/backend/src/database/schema/custodians.ts index 12e0d4c..b46f192 100644 --- a/packages/backend/src/database/schema/custodians.ts +++ b/packages/backend/src/database/schema/custodians.ts @@ -2,10 +2,10 @@ import { pgTable, text, timestamp, uuid } from 'drizzle-orm/pg-core'; import { ingestionProviderEnum } from './ingestion-sources'; export const custodians = pgTable('custodians', { - id: uuid('id').primaryKey().defaultRandom(), - email: text('email').notNull().unique(), - displayName: text('display_name'), - sourceType: ingestionProviderEnum('source_type').notNull(), - createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), - updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow() + id: uuid('id').primaryKey().defaultRandom(), + email: text('email').notNull().unique(), + displayName: text('display_name'), + sourceType: ingestionProviderEnum('source_type').notNull(), + createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), + updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(), }); diff --git a/packages/backend/src/database/schema/ingestion-sources.ts b/packages/backend/src/database/schema/ingestion-sources.ts index 863d9c7..ee326e1 100644 --- a/packages/backend/src/database/schema/ingestion-sources.ts +++ b/packages/backend/src/database/schema/ingestion-sources.ts @@ -1,34 +1,34 @@ import { jsonb, pgEnum, pgTable, text, timestamp, uuid } from 'drizzle-orm/pg-core'; export const ingestionProviderEnum = pgEnum('ingestion_provider', [ - 'google_workspace', - 'microsoft_365', - 'generic_imap', - 'pst_import', - 'eml_import' + 'google_workspace', + 'microsoft_365', + 'generic_imap', + 'pst_import', + 'eml_import', ]); export const ingestionStatusEnum = pgEnum('ingestion_status', [ - 'active', - 'paused', - 'error', - 'pending_auth', - 'syncing', - 'importing', - 'auth_success', - 'imported' + 'active', + 'paused', + 'error', + 'pending_auth', + 'syncing', + 'importing', + 'auth_success', + 'imported', ]); export const ingestionSources = pgTable('ingestion_sources', { - id: uuid('id').primaryKey().defaultRandom(), - name: text('name').notNull(), - provider: ingestionProviderEnum('provider').notNull(), - credentials: text('credentials'), - status: ingestionStatusEnum('status').notNull().default('pending_auth'), - lastSyncStartedAt: timestamp('last_sync_started_at', { withTimezone: true }), - lastSyncFinishedAt: timestamp('last_sync_finished_at', { withTimezone: true }), - lastSyncStatusMessage: text('last_sync_status_message'), - syncState: jsonb('sync_state'), - createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), - updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow() + id: uuid('id').primaryKey().defaultRandom(), + name: text('name').notNull(), + provider: ingestionProviderEnum('provider').notNull(), + credentials: text('credentials'), + status: ingestionStatusEnum('status').notNull().default('pending_auth'), + lastSyncStartedAt: timestamp('last_sync_started_at', { withTimezone: true }), + lastSyncFinishedAt: timestamp('last_sync_finished_at', { withTimezone: true }), + lastSyncStatusMessage: text('last_sync_status_message'), + syncState: jsonb('sync_state'), + createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), + updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(), }); diff --git a/packages/backend/src/database/schema/users.ts b/packages/backend/src/database/schema/users.ts index 9e67661..1a5e6b2 100644 --- a/packages/backend/src/database/schema/users.ts +++ b/packages/backend/src/database/schema/users.ts @@ -1,27 +1,20 @@ import { relations, sql } from 'drizzle-orm'; -import { - pgTable, - text, - timestamp, - uuid, - primaryKey, - jsonb -} from 'drizzle-orm/pg-core'; +import { pgTable, text, timestamp, uuid, primaryKey, jsonb } from 'drizzle-orm/pg-core'; import type { PolicyStatement } from '@open-archiver/types'; /** * The `users` table stores the core user information for authentication and identification. */ export const users = pgTable('users', { - id: uuid('id').primaryKey().defaultRandom(), - email: text('email').notNull().unique(), - first_name: text('first_name'), - last_name: text('last_name'), - password: text('password'), - provider: text('provider').default('local'), - providerId: text('provider_id'), - createdAt: timestamp('created_at').defaultNow().notNull(), - updatedAt: timestamp('updated_at').defaultNow().notNull() + id: uuid('id').primaryKey().defaultRandom(), + email: text('email').notNull().unique(), + first_name: text('first_name'), + last_name: text('last_name'), + password: text('password'), + provider: text('provider').default('local'), + providerId: text('provider_id'), + createdAt: timestamp('created_at').defaultNow().notNull(), + updatedAt: timestamp('updated_at').defaultNow().notNull(), }); /** @@ -29,14 +22,14 @@ export const users = pgTable('users', { * It links a session to a user and records its expiration time. */ export const sessions = pgTable('sessions', { - id: text('id').primaryKey(), - userId: uuid('user_id') - .notNull() - .references(() => users.id, { onDelete: 'cascade' }), - expiresAt: timestamp('expires_at', { - withTimezone: true, - mode: 'date' - }).notNull() + id: text('id').primaryKey(), + userId: uuid('user_id') + .notNull() + .references(() => users.id, { onDelete: 'cascade' }), + expiresAt: timestamp('expires_at', { + withTimezone: true, + mode: 'date', + }).notNull(), }); /** @@ -44,11 +37,14 @@ export const sessions = pgTable('sessions', { * Each role has a name and a set of policies that define its permissions. */ export const roles = pgTable('roles', { - id: uuid('id').primaryKey().defaultRandom(), - name: text('name').notNull().unique(), - policies: jsonb('policies').$type().notNull().default(sql`'[]'::jsonb`), - createdAt: timestamp('created_at').defaultNow().notNull(), - updatedAt: timestamp('updated_at').defaultNow().notNull() + id: uuid('id').primaryKey().defaultRandom(), + name: text('name').notNull().unique(), + policies: jsonb('policies') + .$type() + .notNull() + .default(sql`'[]'::jsonb`), + createdAt: timestamp('created_at').defaultNow().notNull(), + updatedAt: timestamp('updated_at').defaultNow().notNull(), }); /** @@ -56,34 +52,34 @@ export const roles = pgTable('roles', { * This many-to-many relationship allows a user to have multiple roles. */ export const userRoles = pgTable( - 'user_roles', - { - userId: uuid('user_id') - .notNull() - .references(() => users.id, { onDelete: 'cascade' }), - roleId: uuid('role_id') - .notNull() - .references(() => roles.id, { onDelete: 'cascade' }) - }, - (t) => [primaryKey({ columns: [t.userId, t.roleId] })] + 'user_roles', + { + userId: uuid('user_id') + .notNull() + .references(() => users.id, { onDelete: 'cascade' }), + roleId: uuid('role_id') + .notNull() + .references(() => roles.id, { onDelete: 'cascade' }), + }, + (t) => [primaryKey({ columns: [t.userId, t.roleId] })] ); // Define relationships for Drizzle ORM export const usersRelations = relations(users, ({ many }) => ({ - userRoles: many(userRoles) + userRoles: many(userRoles), })); export const rolesRelations = relations(roles, ({ many }) => ({ - userRoles: many(userRoles) + userRoles: many(userRoles), })); export const userRolesRelations = relations(userRoles, ({ one }) => ({ - role: one(roles, { - fields: [userRoles.roleId], - references: [roles.id] - }), - user: one(users, { - fields: [userRoles.userId], - references: [users.id] - }) + role: one(roles, { + fields: [userRoles.roleId], + references: [roles.id], + }), + user: one(users, { + fields: [userRoles.userId], + references: [users.id], + }), })); diff --git a/packages/backend/src/helpers/db.ts b/packages/backend/src/helpers/db.ts index f57c50c..662d261 100644 --- a/packages/backend/src/helpers/db.ts +++ b/packages/backend/src/helpers/db.ts @@ -1,12 +1,12 @@ export const encodeDatabaseUrl = (databaseUrl: string): string => { - try { - const url = new URL(databaseUrl); - if (url.password) { - url.password = encodeURIComponent(url.password); - } - return url.toString(); - } catch (error) { - console.error("Invalid DATABASE_URL, please check your .env file.", error); - throw new Error("Invalid DATABASE_URL"); - } + try { + const url = new URL(databaseUrl); + if (url.password) { + url.password = encodeURIComponent(url.password); + } + return url.toString(); + } catch (error) { + console.error('Invalid DATABASE_URL, please check your .env file.', error); + throw new Error('Invalid DATABASE_URL'); + } }; diff --git a/packages/backend/src/helpers/streamToBuffer.ts b/packages/backend/src/helpers/streamToBuffer.ts index d5ab17a..894cafc 100644 --- a/packages/backend/src/helpers/streamToBuffer.ts +++ b/packages/backend/src/helpers/streamToBuffer.ts @@ -1,8 +1,8 @@ export function streamToBuffer(stream: NodeJS.ReadableStream): Promise { - return new Promise((resolve, reject) => { - const chunks: Buffer[] = []; - stream.on('data', (chunk) => chunks.push(chunk)); - stream.on('error', reject); - stream.on('end', () => resolve(Buffer.concat(chunks))); - }); + return new Promise((resolve, reject) => { + const chunks: Buffer[] = []; + stream.on('data', (chunk) => chunks.push(chunk)); + stream.on('error', reject); + stream.on('end', () => resolve(Buffer.concat(chunks))); + }); } diff --git a/packages/backend/src/helpers/textExtractor.ts b/packages/backend/src/helpers/textExtractor.ts index 48038e7..7a3eb76 100644 --- a/packages/backend/src/helpers/textExtractor.ts +++ b/packages/backend/src/helpers/textExtractor.ts @@ -3,80 +3,68 @@ import mammoth from 'mammoth'; import xlsx from 'xlsx'; function extractTextFromPdf(buffer: Buffer): Promise { - return new Promise((resolve) => { - const pdfParser = new PDFParser(null, true); - let completed = false; + return new Promise((resolve) => { + const pdfParser = new PDFParser(null, true); + let completed = false; - const finish = (text: string) => { - if (completed) return; - completed = true; - pdfParser.removeAllListeners(); - resolve(text); - }; + const finish = (text: string) => { + if (completed) return; + completed = true; + pdfParser.removeAllListeners(); + resolve(text); + }; - pdfParser.on('pdfParser_dataError', () => finish('')); - pdfParser.on('pdfParser_dataReady', () => - finish(pdfParser.getRawTextContent()) - ); + pdfParser.on('pdfParser_dataError', () => finish('')); + pdfParser.on('pdfParser_dataReady', () => finish(pdfParser.getRawTextContent())); - try { - pdfParser.parseBuffer(buffer); - } catch (err) { - console.error('Error parsing PDF buffer', err); - finish(''); - } + try { + pdfParser.parseBuffer(buffer); + } catch (err) { + console.error('Error parsing PDF buffer', err); + finish(''); + } - // Prevent hanging if the parser never emits events - setTimeout(() => finish(''), 10000); - }); + // Prevent hanging if the parser never emits events + setTimeout(() => finish(''), 10000); + }); } -export async function extractText( - buffer: Buffer, - mimeType: string -): Promise { - try { - if (mimeType === 'application/pdf') { - return await extractTextFromPdf(buffer); - } +export async function extractText(buffer: Buffer, mimeType: string): Promise { + try { + if (mimeType === 'application/pdf') { + return await extractTextFromPdf(buffer); + } - if ( - mimeType === - 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' - ) { - const { value } = await mammoth.extractRawText({ buffer }); - return value; - } + if ( + mimeType === 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' + ) { + const { value } = await mammoth.extractRawText({ buffer }); + return value; + } - if ( - mimeType === - 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' - ) { - const workbook = xlsx.read(buffer, { type: 'buffer' }); - let fullText = ''; - for (const sheetName of workbook.SheetNames) { - const sheet = workbook.Sheets[sheetName]; - const sheetText = xlsx.utils.sheet_to_txt(sheet); - fullText += sheetText + '\n'; - } - return fullText; - } + if (mimeType === 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet') { + const workbook = xlsx.read(buffer, { type: 'buffer' }); + let fullText = ''; + for (const sheetName of workbook.SheetNames) { + const sheet = workbook.Sheets[sheetName]; + const sheetText = xlsx.utils.sheet_to_txt(sheet); + fullText += sheetText + '\n'; + } + return fullText; + } - if ( - mimeType.startsWith('text/') || - mimeType === 'application/json' || - mimeType === 'application/xml' - ) { - return buffer.toString('utf-8'); - } - } catch (error) { - console.error( - `Error extracting text from attachment with MIME type ${mimeType}:`, - error - ); - return ''; // Return empty string on failure - } + if ( + mimeType.startsWith('text/') || + mimeType === 'application/json' || + mimeType === 'application/xml' + ) { + return buffer.toString('utf-8'); + } + } catch (error) { + console.error(`Error extracting text from attachment with MIME type ${mimeType}:`, error); + return ''; // Return empty string on failure + } - console.warn(`Unsupported MIME type for text extraction: ${mimeType}`); - return ''; // Return empty string for unsupported types + console.warn(`Unsupported MIME type for text extraction: ${mimeType}`); + return ''; // Return empty string for unsupported types } diff --git a/packages/backend/src/iam-policy/iam-definitions.ts b/packages/backend/src/iam-policy/iam-definitions.ts index bf16cfb..e99bf7f 100644 --- a/packages/backend/src/iam-policy/iam-definitions.ts +++ b/packages/backend/src/iam-policy/iam-definitions.ts @@ -21,71 +21,67 @@ // =================================================================================== const ARCHIVE_ACTIONS = { - READ: 'archive:read', - SEARCH: 'archive:search', - EXPORT: 'archive:export', + READ: 'archive:read', + SEARCH: 'archive:search', + EXPORT: 'archive:export', } as const; const ARCHIVE_RESOURCES = { - ALL: 'archive/all', - INGESTION_SOURCE: 'archive/ingestion-source/*', - MAILBOX: 'archive/mailbox/*', - CUSTODIAN: 'archive/custodian/*', + ALL: 'archive/all', + INGESTION_SOURCE: 'archive/ingestion-source/*', + MAILBOX: 'archive/mailbox/*', + CUSTODIAN: 'archive/custodian/*', } as const; - // =================================================================================== // SERVICE: ingestion // =================================================================================== const INGESTION_ACTIONS = { - CREATE_SOURCE: 'ingestion:createSource', - READ_SOURCE: 'ingestion:readSource', - UPDATE_SOURCE: 'ingestion:updateSource', - DELETE_SOURCE: 'ingestion:deleteSource', - MANAGE_SYNC: 'ingestion:manageSync', // Covers triggering, pausing, and forcing syncs + CREATE_SOURCE: 'ingestion:createSource', + READ_SOURCE: 'ingestion:readSource', + UPDATE_SOURCE: 'ingestion:updateSource', + DELETE_SOURCE: 'ingestion:deleteSource', + MANAGE_SYNC: 'ingestion:manageSync', // Covers triggering, pausing, and forcing syncs } as const; const INGESTION_RESOURCES = { - ALL: 'ingestion-source/*', - SOURCE: 'ingestion-source/{sourceId}', + ALL: 'ingestion-source/*', + SOURCE: 'ingestion-source/{sourceId}', } as const; - // =================================================================================== // SERVICE: system // =================================================================================== const SYSTEM_ACTIONS = { - READ_SETTINGS: 'system:readSettings', - UPDATE_SETTINGS: 'system:updateSettings', - READ_USERS: 'system:readUsers', - CREATE_USER: 'system:createUser', - UPDATE_USER: 'system:updateUser', - DELETE_USER: 'system:deleteUser', - ASSIGN_ROLE: 'system:assignRole', + READ_SETTINGS: 'system:readSettings', + UPDATE_SETTINGS: 'system:updateSettings', + READ_USERS: 'system:readUsers', + CREATE_USER: 'system:createUser', + UPDATE_USER: 'system:updateUser', + DELETE_USER: 'system:deleteUser', + ASSIGN_ROLE: 'system:assignRole', } as const; const SYSTEM_RESOURCES = { - SETTINGS: 'system/settings', - USERS: 'system/users', - USER: 'system/user/{userId}', + SETTINGS: 'system/settings', + USERS: 'system/users', + USER: 'system/user/{userId}', } as const; - // =================================================================================== // SERVICE: dashboard // =================================================================================== const DASHBOARD_ACTIONS = { - READ: 'dashboard:read', + READ: 'dashboard:read', } as const; const DASHBOARD_RESOURCES = { - ALL: 'dashboard/*', + ALL: 'dashboard/*', } as const; - // =================================================================================== // EXPORTED DEFINITIONS // =================================================================================== @@ -95,10 +91,10 @@ const DASHBOARD_RESOURCES = { * This is used by the policy validator to ensure that any action in a policy is recognized. */ export const ValidActions: Set = new Set([ - ...Object.values(ARCHIVE_ACTIONS), - ...Object.values(INGESTION_ACTIONS), - ...Object.values(SYSTEM_ACTIONS), - ...Object.values(DASHBOARD_ACTIONS), + ...Object.values(ARCHIVE_ACTIONS), + ...Object.values(INGESTION_ACTIONS), + ...Object.values(SYSTEM_ACTIONS), + ...Object.values(DASHBOARD_ACTIONS), ]); /** @@ -113,8 +109,8 @@ export const ValidActions: Set = new Set([ * as is `archive/email/123-abc`. */ export const ValidResourcePatterns = { - archive: /^archive\/(all|ingestion-source\/[^\/]+|mailbox\/[^\/]+|custodian\/[^\/]+)$/, - ingestion: /^ingestion-source\/(\*|[^\/]+)$/, - system: /^system\/(settings|users|user\/[^\/]+)$/, - dashboard: /^dashboard\/\*$/, + archive: /^archive\/(all|ingestion-source\/[^\/]+|mailbox\/[^\/]+|custodian\/[^\/]+)$/, + ingestion: /^ingestion-source\/(\*|[^\/]+)$/, + system: /^system\/(settings|users|user\/[^\/]+)$/, + dashboard: /^dashboard\/\*$/, }; diff --git a/packages/backend/src/iam-policy/policy-validator.ts b/packages/backend/src/iam-policy/policy-validator.ts index da910b1..3490ebe 100644 --- a/packages/backend/src/iam-policy/policy-validator.ts +++ b/packages/backend/src/iam-policy/policy-validator.ts @@ -11,90 +11,96 @@ import { ValidActions, ValidResourcePatterns } from './iam-definitions'; * The verification logic is based on the centralized definitions in `iam-definitions.ts`. */ export class PolicyValidator { - /** - * Validates a single policy statement to ensure its actions and resources are valid. - * - * @param {PolicyStatement} statement - The policy statement to validate. - * @returns {{valid: boolean; reason?: string}} - An object containing a boolean `valid` property - * and an optional `reason` string if validation fails. - */ - public static isValid(statement: PolicyStatement): { valid: boolean; reason: string; } { - if (!statement || !statement.Action || !statement.Resource || !statement.Effect) { - return { valid: false, reason: 'Policy statement is missing required fields.' }; - } + /** + * Validates a single policy statement to ensure its actions and resources are valid. + * + * @param {PolicyStatement} statement - The policy statement to validate. + * @returns {{valid: boolean; reason?: string}} - An object containing a boolean `valid` property + * and an optional `reason` string if validation fails. + */ + public static isValid(statement: PolicyStatement): { valid: boolean; reason: string } { + if (!statement || !statement.Action || !statement.Resource || !statement.Effect) { + return { valid: false, reason: 'Policy statement is missing required fields.' }; + } - // 1. Validate Actions - for (const action of statement.Action) { - const { valid, reason } = this.isActionValid(action); - if (!valid) { - return { valid: false, reason }; - } - } + // 1. Validate Actions + for (const action of statement.Action) { + const { valid, reason } = this.isActionValid(action); + if (!valid) { + return { valid: false, reason }; + } + } - // 2. Validate Resources - for (const resource of statement.Resource) { - const { valid, reason } = this.isResourceValid(resource); - if (!valid) { - return { valid: false, reason }; - } - } + // 2. Validate Resources + for (const resource of statement.Resource) { + const { valid, reason } = this.isResourceValid(resource); + if (!valid) { + return { valid: false, reason }; + } + } - return { valid: true, reason: 'valid' }; - } + return { valid: true, reason: 'valid' }; + } - /** - * Checks if a single action string is valid. - * - * Logic: - * - If the action contains a wildcard (e.g., 'archive:*'), it checks if the service part - * (e.g., 'archive') is a recognized service. - * - If there is no wildcard, it checks if the full action string (e.g., 'archive:read') - * exists in the `ValidActions` set. - * - * @param {string} action - The action string to validate. - * @returns {{valid: boolean; reason?: string}} - An object indicating validity and a reason for failure. - */ - private static isActionValid(action: string): { valid: boolean; reason: string; } { - if (action === '*') { - return { valid: true, reason: 'valid' }; - } - if (action.endsWith(':*')) { - const service = action.split(':')[0]; - if (service in ValidResourcePatterns) { - return { valid: true, reason: 'valid' }; - } - return { valid: false, reason: `Invalid service '${service}' in action wildcard '${action}'.` }; - } - if (ValidActions.has(action)) { - return { valid: true, reason: 'valid' }; - } - return { valid: false, reason: `Action '${action}' is not a valid action.` }; - } + /** + * Checks if a single action string is valid. + * + * Logic: + * - If the action contains a wildcard (e.g., 'archive:*'), it checks if the service part + * (e.g., 'archive') is a recognized service. + * - If there is no wildcard, it checks if the full action string (e.g., 'archive:read') + * exists in the `ValidActions` set. + * + * @param {string} action - The action string to validate. + * @returns {{valid: boolean; reason?: string}} - An object indicating validity and a reason for failure. + */ + private static isActionValid(action: string): { valid: boolean; reason: string } { + if (action === '*') { + return { valid: true, reason: 'valid' }; + } + if (action.endsWith(':*')) { + const service = action.split(':')[0]; + if (service in ValidResourcePatterns) { + return { valid: true, reason: 'valid' }; + } + return { + valid: false, + reason: `Invalid service '${service}' in action wildcard '${action}'.`, + }; + } + if (ValidActions.has(action)) { + return { valid: true, reason: 'valid' }; + } + return { valid: false, reason: `Action '${action}' is not a valid action.` }; + } - /** - * Checks if a single resource string has a valid format. - * - * Logic: - * - It extracts the service name from the resource string (e.g., 'archive' from 'archive/all'). - * - It looks up the corresponding regular expression for that service in `ValidResourcePatterns`. - * - It tests the resource string against the pattern. If the service does not exist or the - * pattern does not match, the resource is considered invalid. - * - * @param {string} resource - The resource string to validate. - * @returns {{valid: boolean; reason?: string}} - An object indicating validity and a reason for failure. - */ - private static isResourceValid(resource: string): { valid: boolean; reason: string; } { - const service = resource.split('/')[0]; - if (service === '*') { - return { valid: true, reason: 'valid' }; - } - if (service in ValidResourcePatterns) { - const pattern = ValidResourcePatterns[service as keyof typeof ValidResourcePatterns]; - if (pattern.test(resource)) { - return { valid: true, reason: 'valid' }; - } - return { valid: false, reason: `Resource '${resource}' does not match the expected format for the '${service}' service.` }; - } - return { valid: false, reason: `Invalid service '${service}' in resource '${resource}'.` }; - } + /** + * Checks if a single resource string has a valid format. + * + * Logic: + * - It extracts the service name from the resource string (e.g., 'archive' from 'archive/all'). + * - It looks up the corresponding regular expression for that service in `ValidResourcePatterns`. + * - It tests the resource string against the pattern. If the service does not exist or the + * pattern does not match, the resource is considered invalid. + * + * @param {string} resource - The resource string to validate. + * @returns {{valid: boolean; reason?: string}} - An object indicating validity and a reason for failure. + */ + private static isResourceValid(resource: string): { valid: boolean; reason: string } { + const service = resource.split('/')[0]; + if (service === '*') { + return { valid: true, reason: 'valid' }; + } + if (service in ValidResourcePatterns) { + const pattern = ValidResourcePatterns[service as keyof typeof ValidResourcePatterns]; + if (pattern.test(resource)) { + return { valid: true, reason: 'valid' }; + } + return { + valid: false, + reason: `Resource '${resource}' does not match the expected format for the '${service}' service.`, + }; + } + return { valid: false, reason: `Invalid service '${service}' in resource '${resource}'.` }; + } } diff --git a/packages/backend/src/index.ts b/packages/backend/src/index.ts index da66b16..9b87aff 100644 --- a/packages/backend/src/index.ts +++ b/packages/backend/src/index.ts @@ -22,21 +22,16 @@ import { IamService } from './services/IamService'; import { StorageService } from './services/StorageService'; import { SearchService } from './services/SearchService'; - - // Load environment variables dotenv.config(); // --- Environment Variable Validation --- -const { - PORT_BACKEND, - JWT_SECRET, - JWT_EXPIRES_IN -} = process.env; - +const { PORT_BACKEND, JWT_SECRET, JWT_EXPIRES_IN } = process.env; if (!PORT_BACKEND || !JWT_SECRET || !JWT_EXPIRES_IN) { - throw new Error('Missing required environment variables for the backend: PORT_BACKEND, JWT_SECRET, JWT_EXPIRES_IN.'); + throw new Error( + 'Missing required environment variables for the backend: PORT_BACKEND, JWT_SECRET, JWT_EXPIRES_IN.' + ); } // --- Dependency Injection Setup --- @@ -83,30 +78,30 @@ app.use('/v1/test', testRouter); // Example of a protected route app.get('/v1/protected', requireAuth(authService), (req, res) => { - res.json({ - message: 'You have accessed a protected route!', - user: req.user // The user payload is attached by the requireAuth middleware - }); + res.json({ + message: 'You have accessed a protected route!', + user: req.user, // The user payload is attached by the requireAuth middleware + }); }); -app.get("/", (req, res) => { - res.send('Backend is running!'); +app.get('/', (req, res) => { + res.send('Backend is running!'); }); // --- Server Start --- const startServer = async () => { - try { - // Configure the Meilisearch index on startup - console.log('Configuring email index...'); - await searchService.configureEmailIndex(); + try { + // Configure the Meilisearch index on startup + console.log('Configuring email index...'); + await searchService.configureEmailIndex(); - app.listen(PORT_BACKEND, () => { - console.log(`Backend listening at http://localhost:${PORT_BACKEND}`); - }); - } catch (error) { - console.error('Failed to start the server:', error); - process.exit(1); - } + app.listen(PORT_BACKEND, () => { + console.log(`Backend listening at http://localhost:${PORT_BACKEND}`); + }); + } catch (error) { + console.error('Failed to start the server:', error); + process.exit(1); + } }; startServer(); diff --git a/packages/backend/src/jobs/processors/continuous-sync.processor.ts b/packages/backend/src/jobs/processors/continuous-sync.processor.ts index 7df62de..19d83e0 100644 --- a/packages/backend/src/jobs/processors/continuous-sync.processor.ts +++ b/packages/backend/src/jobs/processors/continuous-sync.processor.ts @@ -6,74 +6,80 @@ import { flowProducer } from '../queues'; import { logger } from '../../config/logger'; export default async (job: Job) => { - const { ingestionSourceId } = job.data; - logger.info({ ingestionSourceId }, 'Starting continuous sync job.'); + const { ingestionSourceId } = job.data; + logger.info({ ingestionSourceId }, 'Starting continuous sync job.'); - const source = await IngestionService.findById(ingestionSourceId); - if (!source || !['error', 'active'].includes(source.status)) { - logger.warn({ ingestionSourceId, status: source?.status }, 'Skipping continuous sync for non-active or non-error source.'); - return; - } + const source = await IngestionService.findById(ingestionSourceId); + if (!source || !['error', 'active'].includes(source.status)) { + logger.warn( + { ingestionSourceId, status: source?.status }, + 'Skipping continuous sync for non-active or non-error source.' + ); + return; + } - await IngestionService.update(ingestionSourceId, { - status: 'syncing', - lastSyncStartedAt: new Date(), - }); + await IngestionService.update(ingestionSourceId, { + status: 'syncing', + lastSyncStartedAt: new Date(), + }); - const connector = EmailProviderFactory.createConnector(source); + const connector = EmailProviderFactory.createConnector(source); - try { - const jobs = []; - for await (const user of connector.listAllUsers()) { - if (user.primaryEmail) { - jobs.push({ - name: 'process-mailbox', - queueName: 'ingestion', - data: { - ingestionSourceId: source.id, - userEmail: user.primaryEmail - }, - opts: { - removeOnComplete: { - age: 60 * 10 // 10 minutes - }, - removeOnFail: { - age: 60 * 30 // 30 minutes - }, - timeout: 1000 * 60 * 30 // 30 minutes - } - }); - } - } - // } + try { + const jobs = []; + for await (const user of connector.listAllUsers()) { + if (user.primaryEmail) { + jobs.push({ + name: 'process-mailbox', + queueName: 'ingestion', + data: { + ingestionSourceId: source.id, + userEmail: user.primaryEmail, + }, + opts: { + removeOnComplete: { + age: 60 * 10, // 10 minutes + }, + removeOnFail: { + age: 60 * 30, // 30 minutes + }, + timeout: 1000 * 60 * 30, // 30 minutes + }, + }); + } + } + // } - if (jobs.length > 0) { - await flowProducer.add({ - name: 'sync-cycle-finished', - queueName: 'ingestion', - data: { - ingestionSourceId, - isInitialImport: false - }, - children: jobs, - opts: { - removeOnComplete: true, - removeOnFail: true - } - }); - } + if (jobs.length > 0) { + await flowProducer.add({ + name: 'sync-cycle-finished', + queueName: 'ingestion', + data: { + ingestionSourceId, + isInitialImport: false, + }, + children: jobs, + opts: { + removeOnComplete: true, + removeOnFail: true, + }, + }); + } - // The status will be set back to 'active' by the 'sync-cycle-finished' job - // once all the mailboxes have been processed. - logger.info({ ingestionSourceId }, 'Continuous sync job finished dispatching mailbox jobs.'); - - } catch (error) { - logger.error({ err: error, ingestionSourceId }, 'Continuous sync job failed.'); - await IngestionService.update(ingestionSourceId, { - status: 'error', - lastSyncFinishedAt: new Date(), - lastSyncStatusMessage: error instanceof Error ? error.message : 'An unknown error occurred during sync.', - }); - throw error; - } + // The status will be set back to 'active' by the 'sync-cycle-finished' job + // once all the mailboxes have been processed. + logger.info( + { ingestionSourceId }, + 'Continuous sync job finished dispatching mailbox jobs.' + ); + } catch (error) { + logger.error({ err: error, ingestionSourceId }, 'Continuous sync job failed.'); + await IngestionService.update(ingestionSourceId, { + status: 'error', + lastSyncFinishedAt: new Date(), + lastSyncStatusMessage: + error instanceof Error ? error.message : 'An unknown error occurred during sync.', + }); + throw error; + } }; diff --git a/packages/backend/src/jobs/processors/index-email.processor.ts b/packages/backend/src/jobs/processors/index-email.processor.ts index 232ff37..d40946e 100644 --- a/packages/backend/src/jobs/processors/index-email.processor.ts +++ b/packages/backend/src/jobs/processors/index-email.processor.ts @@ -9,8 +9,8 @@ const storageService = new StorageService(); const databaseService = new DatabaseService(); const indexingService = new IndexingService(databaseService, searchService, storageService); -export default async function (job: Job<{ emailId: string; }>) { - const { emailId } = job.data; - console.log(`Indexing email with ID: ${emailId}`); - await indexingService.indexEmailById(emailId); +export default async function (job: Job<{ emailId: string }>) { + const { emailId } = job.data; + console.log(`Indexing email with ID: ${emailId}`); + await indexingService.indexEmailById(emailId); } diff --git a/packages/backend/src/jobs/processors/initial-import.processor.ts b/packages/backend/src/jobs/processors/initial-import.processor.ts index adb03a6..e3971db 100644 --- a/packages/backend/src/jobs/processors/initial-import.processor.ts +++ b/packages/backend/src/jobs/processors/initial-import.processor.ts @@ -5,85 +5,89 @@ import { EmailProviderFactory } from '../../services/EmailProviderFactory'; import { flowProducer } from '../queues'; import { logger } from '../../config/logger'; - export default async (job: Job) => { - const { ingestionSourceId } = job.data; - logger.info({ ingestionSourceId }, 'Starting initial import master job'); + const { ingestionSourceId } = job.data; + logger.info({ ingestionSourceId }, 'Starting initial import master job'); - try { - const source = await IngestionService.findById(ingestionSourceId); - if (!source) { - throw new Error(`Ingestion source with ID ${ingestionSourceId} not found`); - } + try { + const source = await IngestionService.findById(ingestionSourceId); + if (!source) { + throw new Error(`Ingestion source with ID ${ingestionSourceId} not found`); + } - await IngestionService.update(ingestionSourceId, { - status: 'importing', - lastSyncStatusMessage: 'Starting initial import...' - }); + await IngestionService.update(ingestionSourceId, { + status: 'importing', + lastSyncStatusMessage: 'Starting initial import...', + }); - const connector = EmailProviderFactory.createConnector(source); + const connector = EmailProviderFactory.createConnector(source); - // if (connector instanceof GoogleWorkspaceConnector || connector instanceof MicrosoftConnector) { - const jobs: FlowChildJob[] = []; - let userCount = 0; - for await (const user of connector.listAllUsers()) { - if (user.primaryEmail) { - jobs.push({ - name: 'process-mailbox', - queueName: 'ingestion', - data: { - ingestionSourceId, - userEmail: user.primaryEmail, - }, - opts: { - removeOnComplete: { - age: 60 * 10 // 10 minutes - }, - removeOnFail: { - age: 60 * 30 // 30 minutes - }, - attempts: 1, - // failParentOnFailure: true - } - }); - userCount++; - } - } + // if (connector instanceof GoogleWorkspaceConnector || connector instanceof MicrosoftConnector) { + const jobs: FlowChildJob[] = []; + let userCount = 0; + for await (const user of connector.listAllUsers()) { + if (user.primaryEmail) { + jobs.push({ + name: 'process-mailbox', + queueName: 'ingestion', + data: { + ingestionSourceId, + userEmail: user.primaryEmail, + }, + opts: { + removeOnComplete: { + age: 60 * 10, // 10 minutes + }, + removeOnFail: { + age: 60 * 30, // 30 minutes + }, + attempts: 1, + // failParentOnFailure: true + }, + }); + userCount++; + } + } - if (jobs.length > 0) { - logger.info({ ingestionSourceId, userCount }, 'Adding sync-cycle-finished job to the queue'); - await flowProducer.add({ - name: 'sync-cycle-finished', - queueName: 'ingestion', - data: { - ingestionSourceId, - userCount, - isInitialImport: true - }, - children: jobs, - opts: { - removeOnComplete: true, - removeOnFail: true - } - }); - } else { - const fileBasedIngestions = IngestionService.returnFileBasedIngestions(); - const finalStatus = fileBasedIngestions.includes(source.provider) ? 'imported' : 'active'; - // If there are no users, we can consider the import finished and set to active - await IngestionService.update(ingestionSourceId, { - status: finalStatus, - lastSyncFinishedAt: new Date(), - lastSyncStatusMessage: 'Initial import complete. No users found.' - }); - } + if (jobs.length > 0) { + logger.info( + { ingestionSourceId, userCount }, + 'Adding sync-cycle-finished job to the queue' + ); + await flowProducer.add({ + name: 'sync-cycle-finished', + queueName: 'ingestion', + data: { + ingestionSourceId, + userCount, + isInitialImport: true, + }, + children: jobs, + opts: { + removeOnComplete: true, + removeOnFail: true, + }, + }); + } else { + const fileBasedIngestions = IngestionService.returnFileBasedIngestions(); + const finalStatus = fileBasedIngestions.includes(source.provider) + ? 'imported' + : 'active'; + // If there are no users, we can consider the import finished and set to active + await IngestionService.update(ingestionSourceId, { + status: finalStatus, + lastSyncFinishedAt: new Date(), + lastSyncStatusMessage: 'Initial import complete. No users found.', + }); + } - logger.info({ ingestionSourceId }, 'Finished initial import master job'); - } catch (error) { - logger.error({ err: error, ingestionSourceId }, 'Error in initial import master job'); - await IngestionService.update(ingestionSourceId, { - status: 'error', - lastSyncStatusMessage: `Initial import failed: ${error instanceof Error ? error.message : 'Unknown error'}` - }); - throw error; - } + logger.info({ ingestionSourceId }, 'Finished initial import master job'); + } catch (error) { + logger.error({ err: error, ingestionSourceId }, 'Error in initial import master job'); + await IngestionService.update(ingestionSourceId, { + status: 'error', + lastSyncStatusMessage: `Initial import failed: ${error instanceof Error ? error.message : 'Unknown error'}`, + }); + throw error; + } }; diff --git a/packages/backend/src/jobs/processors/process-mailbox.processor.ts b/packages/backend/src/jobs/processors/process-mailbox.processor.ts index bc129d0..a183d2a 100644 --- a/packages/backend/src/jobs/processors/process-mailbox.processor.ts +++ b/packages/backend/src/jobs/processors/process-mailbox.processor.ts @@ -14,40 +14,40 @@ import { StorageService } from '../../services/StorageService'; * 'process-mailbox' jobs, aggregating successes, and reporting detailed failures. */ export const processMailboxProcessor = async (job: Job) => { - const { ingestionSourceId, userEmail } = job.data; + const { ingestionSourceId, userEmail } = job.data; - logger.info({ ingestionSourceId, userEmail }, `Processing mailbox for user`); + logger.info({ ingestionSourceId, userEmail }, `Processing mailbox for user`); - try { - const source = await IngestionService.findById(ingestionSourceId); - if (!source) { - throw new Error(`Ingestion source with ID ${ingestionSourceId} not found`); - } + try { + const source = await IngestionService.findById(ingestionSourceId); + if (!source) { + throw new Error(`Ingestion source with ID ${ingestionSourceId} not found`); + } - const connector = EmailProviderFactory.createConnector(source); - const ingestionService = new IngestionService(); - const storageService = new StorageService(); + const connector = EmailProviderFactory.createConnector(source); + const ingestionService = new IngestionService(); + const storageService = new StorageService(); - // Pass the sync state for the entire source, the connector will handle per-user logic if necessary - for await (const email of connector.fetchEmails(userEmail, source.syncState)) { - if (email) { - await ingestionService.processEmail(email, source, storageService, userEmail); - } - } + // Pass the sync state for the entire source, the connector will handle per-user logic if necessary + for await (const email of connector.fetchEmails(userEmail, source.syncState)) { + if (email) { + await ingestionService.processEmail(email, source, storageService, userEmail); + } + } - const newSyncState = connector.getUpdatedSyncState(userEmail); + const newSyncState = connector.getUpdatedSyncState(userEmail); - logger.info({ ingestionSourceId, userEmail }, `Finished processing mailbox for user`); + logger.info({ ingestionSourceId, userEmail }, `Finished processing mailbox for user`); - // Return the new sync state to be aggregated by the parent flow - return newSyncState; - } catch (error) { - logger.error({ err: error, ingestionSourceId, userEmail }, 'Error processing mailbox'); - const errorMessage = error instanceof Error ? error.message : 'An unknown error occurred'; - const processMailboxError: ProcessMailboxError = { - error: true, - message: `Failed to process mailbox for ${userEmail}: ${errorMessage}` - }; - return processMailboxError; - } + // Return the new sync state to be aggregated by the parent flow + return newSyncState; + } catch (error) { + logger.error({ err: error, ingestionSourceId, userEmail }, 'Error processing mailbox'); + const errorMessage = error instanceof Error ? error.message : 'An unknown error occurred'; + const processMailboxError: ProcessMailboxError = { + error: true, + message: `Failed to process mailbox for ${userEmail}: ${errorMessage}`, + }; + return processMailboxError; + } }; diff --git a/packages/backend/src/jobs/processors/schedule-continuous-sync.processor.ts b/packages/backend/src/jobs/processors/schedule-continuous-sync.processor.ts index 5ae425a..23f2b12 100644 --- a/packages/backend/src/jobs/processors/schedule-continuous-sync.processor.ts +++ b/packages/backend/src/jobs/processors/schedule-continuous-sync.processor.ts @@ -5,22 +5,15 @@ import { or, eq } from 'drizzle-orm'; import { ingestionQueue } from '../queues'; export default async (job: Job) => { - console.log( - 'Scheduler running: Looking for active or error ingestion sources to sync.' - ); - // find all sources that have the status of active or error for continuous syncing. - const sourcesToSync = await db - .select({ id: ingestionSources.id }) - .from(ingestionSources) - .where( - or( - eq(ingestionSources.status, 'active'), - eq(ingestionSources.status, 'error') - ) - ); + console.log('Scheduler running: Looking for active or error ingestion sources to sync.'); + // find all sources that have the status of active or error for continuous syncing. + const sourcesToSync = await db + .select({ id: ingestionSources.id }) + .from(ingestionSources) + .where(or(eq(ingestionSources.status, 'active'), eq(ingestionSources.status, 'error'))); - for (const source of sourcesToSync) { - // The status field on the ingestion source is used to prevent duplicate syncs. - await ingestionQueue.add('continuous-sync', { ingestionSourceId: source.id }); - } + for (const source of sourcesToSync) { + // The status field on the ingestion source is used to prevent duplicate syncs. + await ingestionQueue.add('continuous-sync', { ingestionSourceId: source.id }); + } }; diff --git a/packages/backend/src/jobs/processors/sync-cycle-finished.processor.ts b/packages/backend/src/jobs/processors/sync-cycle-finished.processor.ts index d4c3b43..98c17e8 100644 --- a/packages/backend/src/jobs/processors/sync-cycle-finished.processor.ts +++ b/packages/backend/src/jobs/processors/sync-cycle-finished.processor.ts @@ -1,16 +1,21 @@ import { Job } from 'bullmq'; import { IngestionService } from '../../services/IngestionService'; import { logger } from '../../config/logger'; -import { SyncState, ProcessMailboxError, IngestionStatus, IngestionProvider } from '@open-archiver/types'; +import { + SyncState, + ProcessMailboxError, + IngestionStatus, + IngestionProvider, +} from '@open-archiver/types'; import { db } from '../../database'; import { ingestionSources } from '../../database/schema'; import { eq } from 'drizzle-orm'; import { deepmerge } from 'deepmerge-ts'; interface ISyncCycleFinishedJob { - ingestionSourceId: string; - userCount?: number; // Optional, as it's only relevant for the initial import - isInitialImport: boolean; + ingestionSourceId: string; + userCount?: number; // Optional, as it's only relevant for the initial import + isInitialImport: boolean; } /** @@ -28,63 +33,75 @@ interface ISyncCycleFinishedJob { * */ export default async (job: Job) => { - const { ingestionSourceId, userCount, isInitialImport } = job.data; - logger.info({ ingestionSourceId, userCount, isInitialImport }, 'Sync cycle finished job started'); + const { ingestionSourceId, userCount, isInitialImport } = job.data; + logger.info( + { ingestionSourceId, userCount, isInitialImport }, + 'Sync cycle finished job started' + ); - try { - const childrenValues = await job.getChildrenValues(); - const allChildJobs = Object.values(childrenValues); - // if data has error property, it is a failed job - const failedJobs = allChildJobs.filter(v => v && (v as any).error) as ProcessMailboxError[]; - // if data doesn't have error property, it is a successful job with SyncState - const successfulJobs = allChildJobs.filter(v => !v || !(v as any).error) as SyncState[]; + try { + const childrenValues = await job.getChildrenValues(); + const allChildJobs = Object.values(childrenValues); + // if data has error property, it is a failed job + const failedJobs = allChildJobs.filter( + (v) => v && (v as any).error + ) as ProcessMailboxError[]; + // if data doesn't have error property, it is a successful job with SyncState + const successfulJobs = allChildJobs.filter((v) => !v || !(v as any).error) as SyncState[]; - const finalSyncState = deepmerge(...successfulJobs.filter(s => s && Object.keys(s).length > 0)); + const finalSyncState = deepmerge( + ...successfulJobs.filter((s) => s && Object.keys(s).length > 0) + ); - const source = await IngestionService.findById(ingestionSourceId); - let status: IngestionStatus = 'active'; - const fileBasedIngestions = IngestionService.returnFileBasedIngestions(); + const source = await IngestionService.findById(ingestionSourceId); + let status: IngestionStatus = 'active'; + const fileBasedIngestions = IngestionService.returnFileBasedIngestions(); - if (fileBasedIngestions.includes(source.provider)) { - status = 'imported'; - } - let message: string; + if (fileBasedIngestions.includes(source.provider)) { + status = 'imported'; + } + let message: string; - // Check for a specific rate-limit message from the successful jobs - const rateLimitMessage = successfulJobs.find(j => j.statusMessage)?.statusMessage; + // Check for a specific rate-limit message from the successful jobs + const rateLimitMessage = successfulJobs.find((j) => j.statusMessage)?.statusMessage; - if (failedJobs.length > 0) { - status = 'error'; - const errorMessages = failedJobs.map(j => j.message).join('\n'); - message = `Sync cycle completed with ${failedJobs.length} error(s):\n${errorMessages}`; - logger.error({ ingestionSourceId, errors: errorMessages }, 'Sync cycle finished with errors.'); - } else if (rateLimitMessage) { - message = rateLimitMessage; - logger.warn({ ingestionSourceId, message }, 'Sync cycle paused due to rate limiting.'); - } - else { - message = 'Continuous sync cycle finished successfully.'; - if (isInitialImport) { - message = `Initial import finished for ${userCount} mailboxes.`; - } - logger.info({ ingestionSourceId }, 'Successfully updated status and final sync state.'); - } + if (failedJobs.length > 0) { + status = 'error'; + const errorMessages = failedJobs.map((j) => j.message).join('\n'); + message = `Sync cycle completed with ${failedJobs.length} error(s):\n${errorMessages}`; + logger.error( + { ingestionSourceId, errors: errorMessages }, + 'Sync cycle finished with errors.' + ); + } else if (rateLimitMessage) { + message = rateLimitMessage; + logger.warn({ ingestionSourceId, message }, 'Sync cycle paused due to rate limiting.'); + } else { + message = 'Continuous sync cycle finished successfully.'; + if (isInitialImport) { + message = `Initial import finished for ${userCount} mailboxes.`; + } + logger.info({ ingestionSourceId }, 'Successfully updated status and final sync state.'); + } - await db - .update(ingestionSources) - .set({ - status, - lastSyncFinishedAt: new Date(), - lastSyncStatusMessage: message, - syncState: finalSyncState - }) - .where(eq(ingestionSources.id, ingestionSourceId)); - } catch (error) { - logger.error({ err: error, ingestionSourceId }, 'An unexpected error occurred while finalizing the sync cycle.'); - await IngestionService.update(ingestionSourceId, { - status: 'error', - lastSyncFinishedAt: new Date(), - lastSyncStatusMessage: 'An unexpected error occurred while finalizing the sync cycle.' - }); - } + await db + .update(ingestionSources) + .set({ + status, + lastSyncFinishedAt: new Date(), + lastSyncStatusMessage: message, + syncState: finalSyncState, + }) + .where(eq(ingestionSources.id, ingestionSourceId)); + } catch (error) { + logger.error( + { err: error, ingestionSourceId }, + 'An unexpected error occurred while finalizing the sync cycle.' + ); + await IngestionService.update(ingestionSourceId, { + status: 'error', + lastSyncFinishedAt: new Date(), + lastSyncStatusMessage: 'An unexpected error occurred while finalizing the sync cycle.', + }); + } }; diff --git a/packages/backend/src/jobs/queues.ts b/packages/backend/src/jobs/queues.ts index 1e03c80..188fe79 100644 --- a/packages/backend/src/jobs/queues.ts +++ b/packages/backend/src/jobs/queues.ts @@ -5,25 +5,25 @@ export const flowProducer = new FlowProducer({ connection }); // Default job options const defaultJobOptions = { - attempts: 5, - backoff: { - type: 'exponential', - delay: 1000 - }, - removeOnComplete: { - count: 1000 - }, - removeOnFail: { - count: 5000 - } + attempts: 5, + backoff: { + type: 'exponential', + delay: 1000, + }, + removeOnComplete: { + count: 1000, + }, + removeOnFail: { + count: 5000, + }, }; export const ingestionQueue = new Queue('ingestion', { - connection, - defaultJobOptions + connection, + defaultJobOptions, }); export const indexingQueue = new Queue('indexing', { - connection, - defaultJobOptions + connection, + defaultJobOptions, }); diff --git a/packages/backend/src/jobs/schedulers/sync-scheduler.ts b/packages/backend/src/jobs/schedulers/sync-scheduler.ts index 7c27bfb..a3706bf 100644 --- a/packages/backend/src/jobs/schedulers/sync-scheduler.ts +++ b/packages/backend/src/jobs/schedulers/sync-scheduler.ts @@ -3,18 +3,18 @@ import { ingestionQueue } from '../queues'; import { config } from '../../config'; const scheduleContinuousSync = async () => { - // This job will run every 15 minutes - await ingestionQueue.add( - 'schedule-continuous-sync', - {}, - { - repeat: { - pattern: config.app.syncFrequency - }, - } - ); + // This job will run every 15 minutes + await ingestionQueue.add( + 'schedule-continuous-sync', + {}, + { + repeat: { + pattern: config.app.syncFrequency, + }, + } + ); }; scheduleContinuousSync().then(() => { - console.log('Continuous sync scheduler started.'); + console.log('Continuous sync scheduler started.'); }); diff --git a/packages/backend/src/services/ArchivedEmailService.ts b/packages/backend/src/services/ArchivedEmailService.ts index d374479..1084b71 100644 --- a/packages/backend/src/services/ArchivedEmailService.ts +++ b/packages/backend/src/services/ArchivedEmailService.ts @@ -2,215 +2,213 @@ import { count, desc, eq, asc, and } from 'drizzle-orm'; import { db } from '../database'; import { archivedEmails, attachments, emailAttachments } from '../database/schema'; import type { - PaginatedArchivedEmails, - ArchivedEmail, - Recipient, - ThreadEmail, + PaginatedArchivedEmails, + ArchivedEmail, + Recipient, + ThreadEmail, } from '@open-archiver/types'; import { StorageService } from './StorageService'; import { SearchService } from './SearchService'; import type { Readable } from 'stream'; interface DbRecipients { - to: { name: string; address: string; }[]; - cc: { name: string; address: string; }[]; - bcc: { name: string; address: string; }[]; + to: { name: string; address: string }[]; + cc: { name: string; address: string }[]; + bcc: { name: string; address: string }[]; } - - async function streamToBuffer(stream: Readable): Promise { - return new Promise((resolve, reject) => { - const chunks: Buffer[] = []; - stream.on('data', (chunk) => chunks.push(chunk)); - stream.on('error', reject); - stream.on('end', () => resolve(Buffer.concat(chunks))); - }); + return new Promise((resolve, reject) => { + const chunks: Buffer[] = []; + stream.on('data', (chunk) => chunks.push(chunk)); + stream.on('error', reject); + stream.on('end', () => resolve(Buffer.concat(chunks))); + }); } export class ArchivedEmailService { - private static mapRecipients(dbRecipients: unknown): Recipient[] { - const { to = [], cc = [], bcc = [] } = dbRecipients as DbRecipients; + private static mapRecipients(dbRecipients: unknown): Recipient[] { + const { to = [], cc = [], bcc = [] } = dbRecipients as DbRecipients; - const allRecipients = [...to, ...cc, ...bcc]; + const allRecipients = [...to, ...cc, ...bcc]; - return allRecipients.map((r) => ({ - name: r.name, - email: r.address - })); - } + return allRecipients.map((r) => ({ + name: r.name, + email: r.address, + })); + } - public static async getArchivedEmails( - ingestionSourceId: string, - page: number, - limit: number - ): Promise { - const offset = (page - 1) * limit; + public static async getArchivedEmails( + ingestionSourceId: string, + page: number, + limit: number + ): Promise { + const offset = (page - 1) * limit; - const [total] = await db - .select({ - count: count(archivedEmails.id) - }) - .from(archivedEmails) - .where(eq(archivedEmails.ingestionSourceId, ingestionSourceId)); + const [total] = await db + .select({ + count: count(archivedEmails.id), + }) + .from(archivedEmails) + .where(eq(archivedEmails.ingestionSourceId, ingestionSourceId)); - const items = await db - .select() - .from(archivedEmails) - .where(eq(archivedEmails.ingestionSourceId, ingestionSourceId)) - .orderBy(desc(archivedEmails.sentAt)) - .limit(limit) - .offset(offset); + const items = await db + .select() + .from(archivedEmails) + .where(eq(archivedEmails.ingestionSourceId, ingestionSourceId)) + .orderBy(desc(archivedEmails.sentAt)) + .limit(limit) + .offset(offset); - return { - items: items.map((item) => ({ - ...item, - recipients: this.mapRecipients(item.recipients), - tags: (item.tags as string[] | null) || null, - path: item.path || null - })), - total: total.count, - page, - limit - }; - } + return { + items: items.map((item) => ({ + ...item, + recipients: this.mapRecipients(item.recipients), + tags: (item.tags as string[] | null) || null, + path: item.path || null, + })), + total: total.count, + page, + limit, + }; + } - public static async getArchivedEmailById(emailId: string): Promise { - const [email] = await db - .select() - .from(archivedEmails) - .where(eq(archivedEmails.id, emailId)); + public static async getArchivedEmailById(emailId: string): Promise { + const [email] = await db + .select() + .from(archivedEmails) + .where(eq(archivedEmails.id, emailId)); - if (!email) { - return null; - } + if (!email) { + return null; + } - let threadEmails: ThreadEmail[] = []; + let threadEmails: ThreadEmail[] = []; - if (email.threadId) { - threadEmails = await db.query.archivedEmails.findMany({ - where: and( - eq(archivedEmails.threadId, email.threadId), - eq(archivedEmails.ingestionSourceId, email.ingestionSourceId) - ), - orderBy: [asc(archivedEmails.sentAt)], - columns: { - id: true, - subject: true, - sentAt: true, - senderEmail: true, - }, - }); - } + if (email.threadId) { + threadEmails = await db.query.archivedEmails.findMany({ + where: and( + eq(archivedEmails.threadId, email.threadId), + eq(archivedEmails.ingestionSourceId, email.ingestionSourceId) + ), + orderBy: [asc(archivedEmails.sentAt)], + columns: { + id: true, + subject: true, + sentAt: true, + senderEmail: true, + }, + }); + } - const storage = new StorageService(); - const rawStream = await storage.get(email.storagePath); - const raw = await streamToBuffer(rawStream as Readable); + const storage = new StorageService(); + const rawStream = await storage.get(email.storagePath); + const raw = await streamToBuffer(rawStream as Readable); - const mappedEmail = { - ...email, - recipients: this.mapRecipients(email.recipients), - raw, - thread: threadEmails, - tags: (email.tags as string[] | null) || null, - path: email.path || null - }; + const mappedEmail = { + ...email, + recipients: this.mapRecipients(email.recipients), + raw, + thread: threadEmails, + tags: (email.tags as string[] | null) || null, + path: email.path || null, + }; - if (email.hasAttachments) { - const emailAttachmentsResult = await db - .select({ - id: attachments.id, - filename: attachments.filename, - mimeType: attachments.mimeType, - sizeBytes: attachments.sizeBytes, - storagePath: attachments.storagePath - }) - .from(emailAttachments) - .innerJoin(attachments, eq(emailAttachments.attachmentId, attachments.id)) - .where(eq(emailAttachments.emailId, emailId)); + if (email.hasAttachments) { + const emailAttachmentsResult = await db + .select({ + id: attachments.id, + filename: attachments.filename, + mimeType: attachments.mimeType, + sizeBytes: attachments.sizeBytes, + storagePath: attachments.storagePath, + }) + .from(emailAttachments) + .innerJoin(attachments, eq(emailAttachments.attachmentId, attachments.id)) + .where(eq(emailAttachments.emailId, emailId)); - // const attachmentsWithRaw = await Promise.all( - // emailAttachmentsResult.map(async (attachment) => { - // const rawStream = await storage.get(attachment.storagePath); - // const raw = await streamToBuffer(rawStream as Readable); - // return { ...attachment, raw }; - // }) - // ); + // const attachmentsWithRaw = await Promise.all( + // emailAttachmentsResult.map(async (attachment) => { + // const rawStream = await storage.get(attachment.storagePath); + // const raw = await streamToBuffer(rawStream as Readable); + // return { ...attachment, raw }; + // }) + // ); - return { - ...mappedEmail, - attachments: emailAttachmentsResult - }; - } + return { + ...mappedEmail, + attachments: emailAttachmentsResult, + }; + } - return mappedEmail; - } + return mappedEmail; + } - public static async deleteArchivedEmail(emailId: string): Promise { - const [email] = await db - .select() - .from(archivedEmails) - .where(eq(archivedEmails.id, emailId)); + public static async deleteArchivedEmail(emailId: string): Promise { + const [email] = await db + .select() + .from(archivedEmails) + .where(eq(archivedEmails.id, emailId)); - if (!email) { - throw new Error('Archived email not found'); - } + if (!email) { + throw new Error('Archived email not found'); + } - const storage = new StorageService(); + const storage = new StorageService(); - // Load and handle attachments before deleting the email itself - if (email.hasAttachments) { - const emailAttachmentsResult = await db - .select({ - attachmentId: attachments.id, - storagePath: attachments.storagePath, - }) - .from(emailAttachments) - .innerJoin(attachments, eq(emailAttachments.attachmentId, attachments.id)) - .where(eq(emailAttachments.emailId, emailId)); + // Load and handle attachments before deleting the email itself + if (email.hasAttachments) { + const emailAttachmentsResult = await db + .select({ + attachmentId: attachments.id, + storagePath: attachments.storagePath, + }) + .from(emailAttachments) + .innerJoin(attachments, eq(emailAttachments.attachmentId, attachments.id)) + .where(eq(emailAttachments.emailId, emailId)); - try { - for (const attachment of emailAttachmentsResult) { - const [refCount] = await db - .select({ count: count(emailAttachments.emailId) }) - .from(emailAttachments) - .where(eq(emailAttachments.attachmentId, attachment.attachmentId)); + try { + for (const attachment of emailAttachmentsResult) { + const [refCount] = await db + .select({ count: count(emailAttachments.emailId) }) + .from(emailAttachments) + .where(eq(emailAttachments.attachmentId, attachment.attachmentId)); - if (refCount.count === 1) { - await storage.delete(attachment.storagePath); - await db - .delete(emailAttachments) - .where( - and( - eq(emailAttachments.emailId, emailId), - eq(emailAttachments.attachmentId, attachment.attachmentId) - ) - ); - await db - .delete(attachments) - .where(eq(attachments.id, attachment.attachmentId)); - } else { - await db - .delete(emailAttachments) - .where( - and( - eq(emailAttachments.emailId, emailId), - eq(emailAttachments.attachmentId, attachment.attachmentId) - ) - ); - } - } - } catch { - throw new Error('Failed to delete email attachments'); - } - } + if (refCount.count === 1) { + await storage.delete(attachment.storagePath); + await db + .delete(emailAttachments) + .where( + and( + eq(emailAttachments.emailId, emailId), + eq(emailAttachments.attachmentId, attachment.attachmentId) + ) + ); + await db + .delete(attachments) + .where(eq(attachments.id, attachment.attachmentId)); + } else { + await db + .delete(emailAttachments) + .where( + and( + eq(emailAttachments.emailId, emailId), + eq(emailAttachments.attachmentId, attachment.attachmentId) + ) + ); + } + } + } catch { + throw new Error('Failed to delete email attachments'); + } + } - // Delete the email file from storage - await storage.delete(email.storagePath); + // Delete the email file from storage + await storage.delete(email.storagePath); - const searchService = new SearchService(); - await searchService.deleteDocuments('emails', [emailId]); + const searchService = new SearchService(); + await searchService.deleteDocuments('emails', [emailId]); - await db.delete(archivedEmails).where(eq(archivedEmails.id, emailId)); - } + await db.delete(archivedEmails).where(eq(archivedEmails.id, emailId)); + } } diff --git a/packages/backend/src/services/AuthService.ts b/packages/backend/src/services/AuthService.ts index ed331f9..fd52fd1 100644 --- a/packages/backend/src/services/AuthService.ts +++ b/packages/backend/src/services/AuthService.ts @@ -7,72 +7,72 @@ import * as schema from '../database/schema'; import { eq } from 'drizzle-orm'; export class AuthService { - #userService: UserService; - #jwtSecret: Uint8Array; - #jwtExpiresIn: string; + #userService: UserService; + #jwtSecret: Uint8Array; + #jwtExpiresIn: string; - constructor(userService: UserService, jwtSecret: string, jwtExpiresIn: string) { - this.#userService = userService; - this.#jwtSecret = new TextEncoder().encode(jwtSecret); - this.#jwtExpiresIn = jwtExpiresIn; - } + constructor(userService: UserService, jwtSecret: string, jwtExpiresIn: string) { + this.#userService = userService; + this.#jwtSecret = new TextEncoder().encode(jwtSecret); + this.#jwtExpiresIn = jwtExpiresIn; + } - public async verifyPassword(password: string, hash: string): Promise { - return compare(password, hash); - } + public async verifyPassword(password: string, hash: string): Promise { + return compare(password, hash); + } - async #generateAccessToken(payload: AuthTokenPayload): Promise { - if (!payload.sub) { - throw new Error('JWT payload must have a subject (sub) claim.'); - } - return new SignJWT(payload) - .setProtectedHeader({ alg: 'HS256' }) - .setIssuedAt() - .setSubject(payload.sub) - .setExpirationTime(this.#jwtExpiresIn) - .sign(this.#jwtSecret); - } + async #generateAccessToken(payload: AuthTokenPayload): Promise { + if (!payload.sub) { + throw new Error('JWT payload must have a subject (sub) claim.'); + } + return new SignJWT(payload) + .setProtectedHeader({ alg: 'HS256' }) + .setIssuedAt() + .setSubject(payload.sub) + .setExpirationTime(this.#jwtExpiresIn) + .sign(this.#jwtSecret); + } - public async login(email: string, password: string): Promise { - const user = await this.#userService.findByEmail(email); + public async login(email: string, password: string): Promise { + const user = await this.#userService.findByEmail(email); - if (!user || !user.password) { - return null; // User not found or password not set - } + if (!user || !user.password) { + return null; // User not found or password not set + } - const isPasswordValid = await this.verifyPassword(password, user.password); + const isPasswordValid = await this.verifyPassword(password, user.password); - if (!isPasswordValid) { - return null; // Invalid password - } + if (!isPasswordValid) { + return null; // Invalid password + } - const userRoles = await db.query.userRoles.findMany({ - where: eq(schema.userRoles.userId, user.id), - with: { - role: true - } - }); + const userRoles = await db.query.userRoles.findMany({ + where: eq(schema.userRoles.userId, user.id), + with: { + role: true, + }, + }); - const roles = userRoles.map(ur => ur.role.name); + const roles = userRoles.map((ur) => ur.role.name); - const { password: _, ...userWithoutPassword } = user; + const { password: _, ...userWithoutPassword } = user; - const accessToken = await this.#generateAccessToken({ - sub: user.id, - email: user.email, - roles: roles, - }); + const accessToken = await this.#generateAccessToken({ + sub: user.id, + email: user.email, + roles: roles, + }); - return { accessToken, user: userWithoutPassword }; - } + return { accessToken, user: userWithoutPassword }; + } - public async verifyToken(token: string): Promise { - try { - const { payload } = await jwtVerify(token, this.#jwtSecret); - return payload; - } catch (error) { - // Token is invalid or expired - return null; - } - } + public async verifyToken(token: string): Promise { + try { + const { payload } = await jwtVerify(token, this.#jwtSecret); + return payload; + } catch (error) { + // Token is invalid or expired + return null; + } + } } diff --git a/packages/backend/src/services/CryptoService.ts b/packages/backend/src/services/CryptoService.ts index c52865d..11c5318 100644 --- a/packages/backend/src/services/CryptoService.ts +++ b/packages/backend/src/services/CryptoService.ts @@ -8,63 +8,66 @@ const TAG_LENGTH = 16; const ENCRYPTION_KEY = config.app.encryptionKey; if (!ENCRYPTION_KEY) { - throw new Error('ENCRYPTION_KEY is not set in environment variables.'); + throw new Error('ENCRYPTION_KEY is not set in environment variables.'); } // Derive a key from the master encryption key and a salt const getKey = (salt: Buffer): Buffer => { - return scryptSync(ENCRYPTION_KEY, salt, 32); + return scryptSync(ENCRYPTION_KEY, salt, 32); }; export class CryptoService { - public static encrypt(value: string): string { - const salt = randomBytes(SALT_LENGTH); - const key = getKey(salt); - const iv = randomBytes(IV_LENGTH); - const cipher = createCipheriv(ALGORITHM, key, iv); + public static encrypt(value: string): string { + const salt = randomBytes(SALT_LENGTH); + const key = getKey(salt); + const iv = randomBytes(IV_LENGTH); + const cipher = createCipheriv(ALGORITHM, key, iv); - const encrypted = Buffer.concat([cipher.update(value, 'utf8'), cipher.final()]); - const tag = cipher.getAuthTag(); + const encrypted = Buffer.concat([cipher.update(value, 'utf8'), cipher.final()]); + const tag = cipher.getAuthTag(); - return Buffer.concat([salt, iv, tag, encrypted]).toString('hex'); - } + return Buffer.concat([salt, iv, tag, encrypted]).toString('hex'); + } - public static decrypt(encrypted: string): string | null { - try { - const data = Buffer.from(encrypted, 'hex'); - const salt = data.subarray(0, SALT_LENGTH); - const iv = data.subarray(SALT_LENGTH, SALT_LENGTH + IV_LENGTH); - const tag = data.subarray(SALT_LENGTH + IV_LENGTH, SALT_LENGTH + IV_LENGTH + TAG_LENGTH); - const encryptedValue = data.subarray(SALT_LENGTH + IV_LENGTH + TAG_LENGTH); + public static decrypt(encrypted: string): string | null { + try { + const data = Buffer.from(encrypted, 'hex'); + const salt = data.subarray(0, SALT_LENGTH); + const iv = data.subarray(SALT_LENGTH, SALT_LENGTH + IV_LENGTH); + const tag = data.subarray( + SALT_LENGTH + IV_LENGTH, + SALT_LENGTH + IV_LENGTH + TAG_LENGTH + ); + const encryptedValue = data.subarray(SALT_LENGTH + IV_LENGTH + TAG_LENGTH); - const key = getKey(salt); - const decipher = createDecipheriv(ALGORITHM, key, iv); - decipher.setAuthTag(tag); + const key = getKey(salt); + const decipher = createDecipheriv(ALGORITHM, key, iv); + decipher.setAuthTag(tag); - const decrypted = Buffer.concat([decipher.update(encryptedValue), decipher.final()]); + const decrypted = Buffer.concat([decipher.update(encryptedValue), decipher.final()]); - return decrypted.toString('utf8'); - } catch (error) { - console.error('Decryption failed:', error); - return null; - } - } + return decrypted.toString('utf8'); + } catch (error) { + console.error('Decryption failed:', error); + return null; + } + } - public static encryptObject(obj: T): string { - const jsonString = JSON.stringify(obj); - return this.encrypt(jsonString); - } + public static encryptObject(obj: T): string { + const jsonString = JSON.stringify(obj); + return this.encrypt(jsonString); + } - public static decryptObject(encrypted: string): T | null { - const decryptedString = this.decrypt(encrypted); - if (!decryptedString) { - return null; - } - try { - return JSON.parse(decryptedString) as T; - } catch (error) { - console.error('Failed to parse decrypted JSON:', error); - return null; - } - } + public static decryptObject(encrypted: string): T | null { + const decryptedString = this.decrypt(encrypted); + if (!decryptedString) { + return null; + } + try { + return JSON.parse(decryptedString) as T; + } catch (error) { + console.error('Failed to parse decrypted JSON:', error); + return null; + } + } } diff --git a/packages/backend/src/services/DashboardService.ts b/packages/backend/src/services/DashboardService.ts index 473c950..1a03cae 100644 --- a/packages/backend/src/services/DashboardService.ts +++ b/packages/backend/src/services/DashboardService.ts @@ -6,84 +6,84 @@ import { DatabaseService } from './DatabaseService'; import { SearchService } from './SearchService'; class DashboardService { - #db; - #searchService; + #db; + #searchService; - constructor(databaseService: DatabaseService, searchService: SearchService) { - this.#db = databaseService.db; - this.#searchService = searchService; - } + constructor(databaseService: DatabaseService, searchService: SearchService) { + this.#db = databaseService.db; + this.#searchService = searchService; + } - public async getStats() { - const totalEmailsArchived = await this.#db.select({ count: count() }).from(archivedEmails); - const totalStorageUsed = await this.#db - .select({ sum: sql`sum(${archivedEmails.sizeBytes})` }) - .from(archivedEmails); + public async getStats() { + const totalEmailsArchived = await this.#db.select({ count: count() }).from(archivedEmails); + const totalStorageUsed = await this.#db + .select({ sum: sql`sum(${archivedEmails.sizeBytes})` }) + .from(archivedEmails); - const sevenDaysAgo = new Date(); - sevenDaysAgo.setDate(sevenDaysAgo.getDate() - 7); + const sevenDaysAgo = new Date(); + sevenDaysAgo.setDate(sevenDaysAgo.getDate() - 7); - const failedIngestionsLast7Days = await this.#db - .select({ count: count() }) - .from(ingestionSources) - .where( - and( - eq(ingestionSources.status, 'error'), - gte(ingestionSources.updatedAt, sevenDaysAgo) - ) - ); + const failedIngestionsLast7Days = await this.#db + .select({ count: count() }) + .from(ingestionSources) + .where( + and( + eq(ingestionSources.status, 'error'), + gte(ingestionSources.updatedAt, sevenDaysAgo) + ) + ); - return { - totalEmailsArchived: totalEmailsArchived[0].count, - totalStorageUsed: totalStorageUsed[0].sum || 0, - failedIngestionsLast7Days: failedIngestionsLast7Days[0].count - }; - } + return { + totalEmailsArchived: totalEmailsArchived[0].count, + totalStorageUsed: totalStorageUsed[0].sum || 0, + failedIngestionsLast7Days: failedIngestionsLast7Days[0].count, + }; + } - public async getIngestionHistory() { - const thirtyDaysAgo = new Date(); - thirtyDaysAgo.setDate(thirtyDaysAgo.getDate() - 30); + public async getIngestionHistory() { + const thirtyDaysAgo = new Date(); + thirtyDaysAgo.setDate(thirtyDaysAgo.getDate() - 30); - const history = await this.#db - .select({ - date: sql`date_trunc('day', ${archivedEmails.archivedAt})`, - count: count() - }) - .from(archivedEmails) - .where(gte(archivedEmails.archivedAt, thirtyDaysAgo)) - .groupBy(sql`date_trunc('day', ${archivedEmails.archivedAt})`) - .orderBy(sql`date_trunc('day', ${archivedEmails.archivedAt})`); + const history = await this.#db + .select({ + date: sql`date_trunc('day', ${archivedEmails.archivedAt})`, + count: count(), + }) + .from(archivedEmails) + .where(gte(archivedEmails.archivedAt, thirtyDaysAgo)) + .groupBy(sql`date_trunc('day', ${archivedEmails.archivedAt})`) + .orderBy(sql`date_trunc('day', ${archivedEmails.archivedAt})`); - return { history }; - } + return { history }; + } - public async getIngestionSources() { - const sources = await this.#db - .select({ - id: ingestionSources.id, - name: ingestionSources.name, - provider: ingestionSources.provider, - status: ingestionSources.status, - storageUsed: sql`sum(${archivedEmails.sizeBytes})`.mapWith(Number) - }) - .from(ingestionSources) - .leftJoin(archivedEmails, eq(ingestionSources.id, archivedEmails.ingestionSourceId)) - .groupBy(ingestionSources.id); + public async getIngestionSources() { + const sources = await this.#db + .select({ + id: ingestionSources.id, + name: ingestionSources.name, + provider: ingestionSources.provider, + status: ingestionSources.status, + storageUsed: sql`sum(${archivedEmails.sizeBytes})`.mapWith(Number), + }) + .from(ingestionSources) + .leftJoin(archivedEmails, eq(ingestionSources.id, archivedEmails.ingestionSourceId)) + .groupBy(ingestionSources.id); - return sources; - } + return sources; + } - public async getRecentSyncs() { - // This is a placeholder as we don't have a sync job table yet. - return Promise.resolve([]); - } + public async getRecentSyncs() { + // This is a placeholder as we don't have a sync job table yet. + return Promise.resolve([]); + } - public async getIndexedInsights(): Promise { - const topSenders = await this.#searchService.getTopSenders(10); - return { - topSenders - }; - } + public async getIndexedInsights(): Promise { + const topSenders = await this.#searchService.getTopSenders(10); + return { + topSenders, + }; + } } export const dashboardService = new DashboardService(new DatabaseService(), new SearchService()); diff --git a/packages/backend/src/services/DatabaseService.ts b/packages/backend/src/services/DatabaseService.ts index 7f7cb3d..f48fb07 100644 --- a/packages/backend/src/services/DatabaseService.ts +++ b/packages/backend/src/services/DatabaseService.ts @@ -1,6 +1,5 @@ import { db } from '../database'; export class DatabaseService { - public db = db; + public db = db; } - diff --git a/packages/backend/src/services/EmailProviderFactory.ts b/packages/backend/src/services/EmailProviderFactory.ts index a3fe61f..f3cea9b 100644 --- a/packages/backend/src/services/EmailProviderFactory.ts +++ b/packages/backend/src/services/EmailProviderFactory.ts @@ -1,13 +1,13 @@ import type { - IngestionSource, - GoogleWorkspaceCredentials, - Microsoft365Credentials, - GenericImapCredentials, - PSTImportCredentials, - EMLImportCredentials, - EmailObject, - SyncState, - MailboxUser + IngestionSource, + GoogleWorkspaceCredentials, + Microsoft365Credentials, + GenericImapCredentials, + PSTImportCredentials, + EMLImportCredentials, + EmailObject, + SyncState, + MailboxUser, } from '@open-archiver/types'; import { GoogleWorkspaceConnector } from './ingestion-connectors/GoogleWorkspaceConnector'; import { MicrosoftConnector } from './ingestion-connectors/MicrosoftConnector'; @@ -17,31 +17,34 @@ import { EMLConnector } from './ingestion-connectors/EMLConnector'; // Define a common interface for all connectors export interface IEmailConnector { - testConnection(): Promise; - fetchEmails(userEmail: string, syncState?: SyncState | null): AsyncGenerator; - getUpdatedSyncState(userEmail?: string): SyncState; - listAllUsers(): AsyncGenerator; - returnImapUserEmail?(): string; + testConnection(): Promise; + fetchEmails( + userEmail: string, + syncState?: SyncState | null + ): AsyncGenerator; + getUpdatedSyncState(userEmail?: string): SyncState; + listAllUsers(): AsyncGenerator; + returnImapUserEmail?(): string; } export class EmailProviderFactory { - static createConnector(source: IngestionSource): IEmailConnector { - // Credentials are now decrypted by the IngestionService before being passed around - const credentials = source.credentials; + static createConnector(source: IngestionSource): IEmailConnector { + // Credentials are now decrypted by the IngestionService before being passed around + const credentials = source.credentials; - switch (source.provider) { - case 'google_workspace': - return new GoogleWorkspaceConnector(credentials as GoogleWorkspaceCredentials); - case 'microsoft_365': - return new MicrosoftConnector(credentials as Microsoft365Credentials); - case 'generic_imap': - return new ImapConnector(credentials as GenericImapCredentials); - case 'pst_import': - return new PSTConnector(credentials as PSTImportCredentials); - case 'eml_import': - return new EMLConnector(credentials as EMLImportCredentials); - default: - throw new Error(`Unsupported provider: ${source.provider}`); - } - } + switch (source.provider) { + case 'google_workspace': + return new GoogleWorkspaceConnector(credentials as GoogleWorkspaceCredentials); + case 'microsoft_365': + return new MicrosoftConnector(credentials as Microsoft365Credentials); + case 'generic_imap': + return new ImapConnector(credentials as GenericImapCredentials); + case 'pst_import': + return new PSTConnector(credentials as PSTImportCredentials); + case 'eml_import': + return new EMLConnector(credentials as EMLImportCredentials); + default: + throw new Error(`Unsupported provider: ${source.provider}`); + } + } } diff --git a/packages/backend/src/services/IamService.ts b/packages/backend/src/services/IamService.ts index eec25d5..1489f03 100644 --- a/packages/backend/src/services/IamService.ts +++ b/packages/backend/src/services/IamService.ts @@ -4,21 +4,21 @@ import type { Role, PolicyStatement } from '@open-archiver/types'; import { eq } from 'drizzle-orm'; export class IamService { - public async getRoles(): Promise { - return db.select().from(roles); - } + public async getRoles(): Promise { + return db.select().from(roles); + } - public async getRoleById(id: string): Promise { - const [role] = await db.select().from(roles).where(eq(roles.id, id)); - return role; - } + public async getRoleById(id: string): Promise { + const [role] = await db.select().from(roles).where(eq(roles.id, id)); + return role; + } - public async createRole(name: string, policy: PolicyStatement[]): Promise { - const [role] = await db.insert(roles).values({ name, policies: policy }).returning(); - return role; - } + public async createRole(name: string, policy: PolicyStatement[]): Promise { + const [role] = await db.insert(roles).values({ name, policies: policy }).returning(); + return role; + } - public async deleteRole(id: string): Promise { - await db.delete(roles).where(eq(roles.id, id)); - } + public async deleteRole(id: string): Promise { + await db.delete(roles).where(eq(roles.id, id)); + } } diff --git a/packages/backend/src/services/IndexingService.ts b/packages/backend/src/services/IndexingService.ts index ca81226..6567d72 100644 --- a/packages/backend/src/services/IndexingService.ts +++ b/packages/backend/src/services/IndexingService.ts @@ -9,192 +9,191 @@ import { streamToBuffer } from '../helpers/streamToBuffer'; import { simpleParser } from 'mailparser'; interface DbRecipients { - to: { name: string; address: string; }[]; - cc: { name: string; address: string; }[]; - bcc: { name: string; address: string; }[]; + to: { name: string; address: string }[]; + cc: { name: string; address: string }[]; + bcc: { name: string; address: string }[]; } type AttachmentsType = { - filename: string, - buffer: Buffer, - mimeType: string; + filename: string; + buffer: Buffer; + mimeType: string; }[]; export class IndexingService { - private dbService: DatabaseService; - private searchService: SearchService; - private storageService: StorageService; + private dbService: DatabaseService; + private searchService: SearchService; + private storageService: StorageService; - /** - * Initializes the service with its dependencies. - */ - constructor( - dbService: DatabaseService, - searchService: SearchService, - storageService: StorageService, - ) { - this.dbService = dbService; - this.searchService = searchService; - this.storageService = storageService; - } + /** + * Initializes the service with its dependencies. + */ + constructor( + dbService: DatabaseService, + searchService: SearchService, + storageService: StorageService + ) { + this.dbService = dbService; + this.searchService = searchService; + this.storageService = storageService; + } - /** - * Fetches an email by its ID from the database, creates a search document, and indexes it. - */ - public async indexEmailById(emailId: string): Promise { - const email = await this.dbService.db.query.archivedEmails.findFirst({ - where: eq(archivedEmails.id, emailId), - }); + /** + * Fetches an email by its ID from the database, creates a search document, and indexes it. + */ + public async indexEmailById(emailId: string): Promise { + const email = await this.dbService.db.query.archivedEmails.findFirst({ + where: eq(archivedEmails.id, emailId), + }); - if (!email) { - throw new Error(`Email with ID ${emailId} not found for indexing.`); - } + if (!email) { + throw new Error(`Email with ID ${emailId} not found for indexing.`); + } - let emailAttachmentsResult: Attachment[] = []; - if (email.hasAttachments) { - emailAttachmentsResult = await this.dbService.db - .select({ - id: attachments.id, - filename: attachments.filename, - mimeType: attachments.mimeType, - sizeBytes: attachments.sizeBytes, - contentHashSha256: attachments.contentHashSha256, - storagePath: attachments.storagePath, - }) - .from(emailAttachments) - .innerJoin(attachments, eq(emailAttachments.attachmentId, attachments.id)) - .where(eq(emailAttachments.emailId, emailId)); - } + let emailAttachmentsResult: Attachment[] = []; + if (email.hasAttachments) { + emailAttachmentsResult = await this.dbService.db + .select({ + id: attachments.id, + filename: attachments.filename, + mimeType: attachments.mimeType, + sizeBytes: attachments.sizeBytes, + contentHashSha256: attachments.contentHashSha256, + storagePath: attachments.storagePath, + }) + .from(emailAttachments) + .innerJoin(attachments, eq(emailAttachments.attachmentId, attachments.id)) + .where(eq(emailAttachments.emailId, emailId)); + } - const document = await this.createEmailDocument(email, emailAttachmentsResult); - await this.searchService.addDocuments('emails', [document], 'id'); - } + const document = await this.createEmailDocument(email, emailAttachmentsResult); + await this.searchService.addDocuments('emails', [document], 'id'); + } - /** - * Indexes an email object directly, creates a search document, and indexes it. - */ - public async indexByEmail(email: EmailObject, ingestionSourceId: string, archivedEmailId: string): Promise { - const attachments: AttachmentsType = []; - if (email.attachments && email.attachments.length > 0) { - for (const attachment of email.attachments) { - attachments.push({ - buffer: attachment.content, - filename: attachment.filename, - mimeType: attachment.contentType - }); - } - } - const document = await this.createEmailDocumentFromRaw(email, attachments, ingestionSourceId, archivedEmailId); - await this.searchService.addDocuments('emails', [document], 'id'); - } + /** + * Indexes an email object directly, creates a search document, and indexes it. + */ + public async indexByEmail( + email: EmailObject, + ingestionSourceId: string, + archivedEmailId: string + ): Promise { + const attachments: AttachmentsType = []; + if (email.attachments && email.attachments.length > 0) { + for (const attachment of email.attachments) { + attachments.push({ + buffer: attachment.content, + filename: attachment.filename, + mimeType: attachment.contentType, + }); + } + } + const document = await this.createEmailDocumentFromRaw( + email, + attachments, + ingestionSourceId, + archivedEmailId + ); + await this.searchService.addDocuments('emails', [document], 'id'); + } - /** - * Creates a search document from a raw email object and its attachments. - */ - private async createEmailDocumentFromRaw( - email: EmailObject, - attachments: AttachmentsType, - ingestionSourceId: string, - archivedEmailId: string - ): Promise { - const extractedAttachments = []; - for (const attachment of attachments) { - try { - const textContent = await extractText( - attachment.buffer, - attachment.mimeType || '' - ); - extractedAttachments.push({ - filename: attachment.filename, - content: textContent, - }); - } catch (error) { - console.error( - `Failed to extract text from attachment: ${attachment.filename}`, - error - ); - // skip attachment or fail the job - } - } - return { - id: archivedEmailId, - from: email.from[0]?.address, - to: email.to.map((i: EmailAddress) => i.address) || [], - cc: email.cc?.map((i: EmailAddress) => i.address) || [], - bcc: email.bcc?.map((i: EmailAddress) => i.address) || [], - subject: email.subject || '', - body: email.body || email.html || '', - attachments: extractedAttachments, - timestamp: new Date(email.receivedAt).getTime(), - ingestionSourceId: ingestionSourceId - }; - } + /** + * Creates a search document from a raw email object and its attachments. + */ + private async createEmailDocumentFromRaw( + email: EmailObject, + attachments: AttachmentsType, + ingestionSourceId: string, + archivedEmailId: string + ): Promise { + const extractedAttachments = []; + for (const attachment of attachments) { + try { + const textContent = await extractText(attachment.buffer, attachment.mimeType || ''); + extractedAttachments.push({ + filename: attachment.filename, + content: textContent, + }); + } catch (error) { + console.error( + `Failed to extract text from attachment: ${attachment.filename}`, + error + ); + // skip attachment or fail the job + } + } + return { + id: archivedEmailId, + from: email.from[0]?.address, + to: email.to.map((i: EmailAddress) => i.address) || [], + cc: email.cc?.map((i: EmailAddress) => i.address) || [], + bcc: email.bcc?.map((i: EmailAddress) => i.address) || [], + subject: email.subject || '', + body: email.body || email.html || '', + attachments: extractedAttachments, + timestamp: new Date(email.receivedAt).getTime(), + ingestionSourceId: ingestionSourceId, + }; + } + /** + * Creates a search document from a database email record and its attachments. + */ + private async createEmailDocument( + email: typeof archivedEmails.$inferSelect, + attachments: Attachment[] + ): Promise { + const attachmentContents = await this.extractAttachmentContents(attachments); - /** - * Creates a search document from a database email record and its attachments. - */ - private async createEmailDocument( - email: typeof archivedEmails.$inferSelect, - attachments: Attachment[] - ): Promise { - const attachmentContents = await this.extractAttachmentContents(attachments); + const emailBodyStream = await this.storageService.get(email.storagePath); + const emailBodyBuffer = await streamToBuffer(emailBodyStream); + const parsedEmail = await simpleParser(emailBodyBuffer); + const emailBodyText = + parsedEmail.text || + parsedEmail.html || + (await extractText(emailBodyBuffer, 'text/plain')) || + ''; - const emailBodyStream = await this.storageService.get(email.storagePath); - const emailBodyBuffer = await streamToBuffer(emailBodyStream); - const parsedEmail = await simpleParser(emailBodyBuffer); - const emailBodyText = - parsedEmail.text || - parsedEmail.html || - (await extractText(emailBodyBuffer, 'text/plain')) || - ''; + const recipients = email.recipients as DbRecipients; - const recipients = email.recipients as DbRecipients; - - return { - id: email.id, - from: email.senderEmail, - to: recipients.to?.map((r) => r.address) || [], - cc: recipients.cc?.map((r) => r.address) || [], - bcc: recipients.bcc?.map((r) => r.address) || [], - subject: email.subject || '', - body: emailBodyText, - attachments: attachmentContents, - timestamp: new Date(email.sentAt).getTime(), - ingestionSourceId: email.ingestionSourceId - }; - } - - /** - * Extracts text content from a list of attachments. - */ - private async extractAttachmentContents( - attachments: Attachment[] - ): Promise<{ filename: string; content: string; }[]> { - const extractedAttachments = []; - for (const attachment of attachments) { - try { - const fileStream = await this.storageService.get( - attachment.storagePath - ); - const fileBuffer = await streamToBuffer(fileStream); - const textContent = await extractText( - fileBuffer, - attachment.mimeType || '' - ); - extractedAttachments.push({ - filename: attachment.filename, - content: textContent, - }); - } catch (error) { - console.error( - `Failed to extract text from attachment: ${attachment.filename}`, - error - ); - // skip attachment or fail the job - } - } - return extractedAttachments; - } + return { + id: email.id, + from: email.senderEmail, + to: recipients.to?.map((r) => r.address) || [], + cc: recipients.cc?.map((r) => r.address) || [], + bcc: recipients.bcc?.map((r) => r.address) || [], + subject: email.subject || '', + body: emailBodyText, + attachments: attachmentContents, + timestamp: new Date(email.sentAt).getTime(), + ingestionSourceId: email.ingestionSourceId, + }; + } + /** + * Extracts text content from a list of attachments. + */ + private async extractAttachmentContents( + attachments: Attachment[] + ): Promise<{ filename: string; content: string }[]> { + const extractedAttachments = []; + for (const attachment of attachments) { + try { + const fileStream = await this.storageService.get(attachment.storagePath); + const fileBuffer = await streamToBuffer(fileStream); + const textContent = await extractText(fileBuffer, attachment.mimeType || ''); + extractedAttachments.push({ + filename: attachment.filename, + content: textContent, + }); + } catch (error) { + console.error( + `Failed to extract text from attachment: ${attachment.filename}`, + error + ); + // skip attachment or fail the job + } + } + return extractedAttachments; + } } diff --git a/packages/backend/src/services/IngestionService.ts b/packages/backend/src/services/IngestionService.ts index 156cb76..c423b60 100644 --- a/packages/backend/src/services/IngestionService.ts +++ b/packages/backend/src/services/IngestionService.ts @@ -1,11 +1,11 @@ import { db } from '../database'; import { ingestionSources } from '../database/schema'; import type { - CreateIngestionSourceDto, - UpdateIngestionSourceDto, - IngestionSource, - IngestionCredentials, - IngestionProvider + CreateIngestionSourceDto, + UpdateIngestionSourceDto, + IngestionSource, + IngestionCredentials, + IngestionProvider, } from '@open-archiver/types'; import { and, desc, eq } from 'drizzle-orm'; import { CryptoService } from './CryptoService'; @@ -14,7 +14,11 @@ import { ingestionQueue } from '../jobs/queues'; import type { JobType } from 'bullmq'; import { StorageService } from './StorageService'; import type { IInitialImportJob, EmailObject } from '@open-archiver/types'; -import { archivedEmails, attachments as attachmentsSchema, emailAttachments } from '../database/schema'; +import { + archivedEmails, + attachments as attachmentsSchema, + emailAttachments, +} from '../database/schema'; import { createHash } from 'crypto'; import { logger } from '../config/logger'; import { IndexingService } from './IndexingService'; @@ -22,352 +26,386 @@ import { SearchService } from './SearchService'; import { DatabaseService } from './DatabaseService'; import { config } from '../config/index'; - export class IngestionService { - private static decryptSource(source: typeof ingestionSources.$inferSelect): IngestionSource | null { - const decryptedCredentials = CryptoService.decryptObject( - source.credentials as string - ); + private static decryptSource( + source: typeof ingestionSources.$inferSelect + ): IngestionSource | null { + const decryptedCredentials = CryptoService.decryptObject( + source.credentials as string + ); - if (!decryptedCredentials) { - logger.error({ sourceId: source.id }, 'Failed to decrypt ingestion source credentials.'); - return null; - } + if (!decryptedCredentials) { + logger.error( + { sourceId: source.id }, + 'Failed to decrypt ingestion source credentials.' + ); + return null; + } - return { ...source, credentials: decryptedCredentials } as IngestionSource; - } + return { ...source, credentials: decryptedCredentials } as IngestionSource; + } - public static returnFileBasedIngestions(): IngestionProvider[] { - return ['pst_import', 'eml_import']; - } + public static returnFileBasedIngestions(): IngestionProvider[] { + return ['pst_import', 'eml_import']; + } - public static async create(dto: CreateIngestionSourceDto): Promise { - const { providerConfig, ...rest } = dto; - const encryptedCredentials = CryptoService.encryptObject(providerConfig); + public static async create(dto: CreateIngestionSourceDto): Promise { + const { providerConfig, ...rest } = dto; + const encryptedCredentials = CryptoService.encryptObject(providerConfig); - const valuesToInsert = { - ...rest, - status: 'pending_auth' as const, - credentials: encryptedCredentials - }; + const valuesToInsert = { + ...rest, + status: 'pending_auth' as const, + credentials: encryptedCredentials, + }; - const [newSource] = await db.insert(ingestionSources).values(valuesToInsert).returning(); + const [newSource] = await db.insert(ingestionSources).values(valuesToInsert).returning(); - const decryptedSource = this.decryptSource(newSource); - if (!decryptedSource) { - await this.delete(newSource.id); - throw new Error('Failed to process newly created ingestion source due to a decryption error.'); - } - const connector = EmailProviderFactory.createConnector(decryptedSource); + const decryptedSource = this.decryptSource(newSource); + if (!decryptedSource) { + await this.delete(newSource.id); + throw new Error( + 'Failed to process newly created ingestion source due to a decryption error.' + ); + } + const connector = EmailProviderFactory.createConnector(decryptedSource); - try { - await connector.testConnection(); - // If connection succeeds, update status to auth_success, which triggers the initial import. - return await this.update(decryptedSource.id, { status: 'auth_success' }); - } catch (error) { - // If connection fails, delete the newly created source and throw the error. - await this.delete(decryptedSource.id); - throw error; - } - } + try { + await connector.testConnection(); + // If connection succeeds, update status to auth_success, which triggers the initial import. + return await this.update(decryptedSource.id, { status: 'auth_success' }); + } catch (error) { + // If connection fails, delete the newly created source and throw the error. + await this.delete(decryptedSource.id); + throw error; + } + } - public static async findAll(): Promise { - const sources = await db.select().from(ingestionSources).orderBy(desc(ingestionSources.createdAt)); - return sources.flatMap(source => { - const decrypted = this.decryptSource(source); - return decrypted ? [decrypted] : []; - }); - } + public static async findAll(): Promise { + const sources = await db + .select() + .from(ingestionSources) + .orderBy(desc(ingestionSources.createdAt)); + return sources.flatMap((source) => { + const decrypted = this.decryptSource(source); + return decrypted ? [decrypted] : []; + }); + } - public static async findById(id: string): Promise { - const [source] = await db.select().from(ingestionSources).where(eq(ingestionSources.id, id)); - if (!source) { - throw new Error('Ingestion source not found'); - } - const decryptedSource = this.decryptSource(source); - if (!decryptedSource) { - throw new Error('Failed to decrypt ingestion source credentials.'); - } - return decryptedSource; - } + public static async findById(id: string): Promise { + const [source] = await db + .select() + .from(ingestionSources) + .where(eq(ingestionSources.id, id)); + if (!source) { + throw new Error('Ingestion source not found'); + } + const decryptedSource = this.decryptSource(source); + if (!decryptedSource) { + throw new Error('Failed to decrypt ingestion source credentials.'); + } + return decryptedSource; + } - public static async update( - id: string, - dto: UpdateIngestionSourceDto - ): Promise { - const { providerConfig, ...rest } = dto; - const valuesToUpdate: Partial = { ...rest }; + public static async update( + id: string, + dto: UpdateIngestionSourceDto + ): Promise { + const { providerConfig, ...rest } = dto; + const valuesToUpdate: Partial = { ...rest }; - // Get the original source to compare the status later - const originalSource = await this.findById(id); + // Get the original source to compare the status later + const originalSource = await this.findById(id); - if (providerConfig) { - // Encrypt the new credentials before updating - valuesToUpdate.credentials = CryptoService.encryptObject(providerConfig); - } + if (providerConfig) { + // Encrypt the new credentials before updating + valuesToUpdate.credentials = CryptoService.encryptObject(providerConfig); + } - const [updatedSource] = await db - .update(ingestionSources) - .set(valuesToUpdate) - .where(eq(ingestionSources.id, id)) - .returning(); + const [updatedSource] = await db + .update(ingestionSources) + .set(valuesToUpdate) + .where(eq(ingestionSources.id, id)) + .returning(); - if (!updatedSource) { - throw new Error('Ingestion source not found'); - } + if (!updatedSource) { + throw new Error('Ingestion source not found'); + } - const decryptedSource = this.decryptSource(updatedSource); + const decryptedSource = this.decryptSource(updatedSource); - if (!decryptedSource) { - throw new Error('Failed to process updated ingestion source due to a decryption error.'); - } + if (!decryptedSource) { + throw new Error( + 'Failed to process updated ingestion source due to a decryption error.' + ); + } - // If the status has changed to auth_success, trigger the initial import - if ( - originalSource.status !== 'auth_success' && - decryptedSource.status === 'auth_success' - ) { - await this.triggerInitialImport(decryptedSource.id); - } + // If the status has changed to auth_success, trigger the initial import + if (originalSource.status !== 'auth_success' && decryptedSource.status === 'auth_success') { + await this.triggerInitialImport(decryptedSource.id); + } - return decryptedSource; - } + return decryptedSource; + } - public static async delete(id: string): Promise { - const source = await this.findById(id); - if (!source) { - throw new Error('Ingestion source not found'); - } + public static async delete(id: string): Promise { + const source = await this.findById(id); + if (!source) { + throw new Error('Ingestion source not found'); + } - // Delete all emails and attachments from storage - const storage = new StorageService(); - const emailPath = `${config.storage.openArchiverFolderName}/${source.name.replaceAll(' ', '-')}-${source.id}/`; - await storage.delete(emailPath); + // Delete all emails and attachments from storage + const storage = new StorageService(); + const emailPath = `${config.storage.openArchiverFolderName}/${source.name.replaceAll(' ', '-')}-${source.id}/`; + await storage.delete(emailPath); - if ( - (source.credentials.type === 'pst_import' || source.credentials.type === 'eml_import') && - source.credentials.uploadedFilePath && - (await storage.exists(source.credentials.uploadedFilePath)) - ) { - await storage.delete(source.credentials.uploadedFilePath); - } + if ( + (source.credentials.type === 'pst_import' || + source.credentials.type === 'eml_import') && + source.credentials.uploadedFilePath && + (await storage.exists(source.credentials.uploadedFilePath)) + ) { + await storage.delete(source.credentials.uploadedFilePath); + } - // Delete all emails from the database - // NOTE: This is done by database CASADE, change when CASADE relation no longer exists. - // await db.delete(archivedEmails).where(eq(archivedEmails.ingestionSourceId, id)); + // Delete all emails from the database + // NOTE: This is done by database CASADE, change when CASADE relation no longer exists. + // await db.delete(archivedEmails).where(eq(archivedEmails.ingestionSourceId, id)); - // Delete all documents from Meilisearch - const searchService = new SearchService(); - await searchService.deleteDocumentsByFilter('emails', `ingestionSourceId = ${id}`); + // Delete all documents from Meilisearch + const searchService = new SearchService(); + await searchService.deleteDocumentsByFilter('emails', `ingestionSourceId = ${id}`); - const [deletedSource] = await db - .delete(ingestionSources) - .where(eq(ingestionSources.id, id)) - .returning(); + const [deletedSource] = await db + .delete(ingestionSources) + .where(eq(ingestionSources.id, id)) + .returning(); - const decryptedSource = this.decryptSource(deletedSource); - if (!decryptedSource) { - // Even if decryption fails, we should confirm deletion. - // We might return a simpler object or just a success message. - // For now, we'll indicate the issue but still confirm deletion happened. - logger.warn({ sourceId: deletedSource.id }, 'Could not decrypt credentials of deleted source, but deletion was successful.'); - return { ...deletedSource, credentials: null } as unknown as IngestionSource; - } - return decryptedSource; - } + const decryptedSource = this.decryptSource(deletedSource); + if (!decryptedSource) { + // Even if decryption fails, we should confirm deletion. + // We might return a simpler object or just a success message. + // For now, we'll indicate the issue but still confirm deletion happened. + logger.warn( + { sourceId: deletedSource.id }, + 'Could not decrypt credentials of deleted source, but deletion was successful.' + ); + return { ...deletedSource, credentials: null } as unknown as IngestionSource; + } + return decryptedSource; + } - public static async triggerInitialImport(id: string): Promise { - const source = await this.findById(id); + public static async triggerInitialImport(id: string): Promise { + const source = await this.findById(id); - await ingestionQueue.add('initial-import', { ingestionSourceId: source.id }); + await ingestionQueue.add('initial-import', { ingestionSourceId: source.id }); + } - } + public static async triggerForceSync(id: string): Promise { + const source = await this.findById(id); + logger.info({ ingestionSourceId: id }, 'Force syncing started.'); + if (!source) { + throw new Error('Ingestion source not found'); + } - public static async triggerForceSync(id: string): Promise { - const source = await this.findById(id); - logger.info({ ingestionSourceId: id }, 'Force syncing started.'); - if (!source) { - throw new Error('Ingestion source not found'); - } + // Clean up existing jobs for this source to break any stuck flows + const jobTypes: JobType[] = ['active', 'waiting', 'failed', 'delayed', 'paused']; + const jobs = await ingestionQueue.getJobs(jobTypes); + for (const job of jobs) { + if (job.data.ingestionSourceId === id) { + try { + await job.remove(); + logger.info( + { jobId: job.id, ingestionSourceId: id }, + 'Removed stale job during force sync.' + ); + } catch (error) { + logger.error({ err: error, jobId: job.id }, 'Failed to remove stale job.'); + } + } + } - // Clean up existing jobs for this source to break any stuck flows - const jobTypes: JobType[] = ['active', 'waiting', 'failed', 'delayed', 'paused']; - const jobs = await ingestionQueue.getJobs(jobTypes); - for (const job of jobs) { - if (job.data.ingestionSourceId === id) { - try { - await job.remove(); - logger.info({ jobId: job.id, ingestionSourceId: id }, 'Removed stale job during force sync.'); - } catch (error) { - logger.error({ err: error, jobId: job.id }, 'Failed to remove stale job.'); - } - } - } + // Reset status to 'active' + await this.update(id, { + status: 'active', + lastSyncStatusMessage: 'Force sync triggered by user.', + }); - // Reset status to 'active' - await this.update(id, { status: 'active', lastSyncStatusMessage: 'Force sync triggered by user.' }); + await ingestionQueue.add('continuous-sync', { ingestionSourceId: source.id }); + } + public async performBulkImport(job: IInitialImportJob): Promise { + const { ingestionSourceId } = job; + const source = await IngestionService.findById(ingestionSourceId); + if (!source) { + throw new Error(`Ingestion source ${ingestionSourceId} not found.`); + } - await ingestionQueue.add('continuous-sync', { ingestionSourceId: source.id }); - } + logger.info(`Starting bulk import for source: ${source.name} (${source.id})`); + await IngestionService.update(ingestionSourceId, { + status: 'importing', + lastSyncStartedAt: new Date(), + }); - public async performBulkImport(job: IInitialImportJob): Promise { - const { ingestionSourceId } = job; - const source = await IngestionService.findById(ingestionSourceId); - if (!source) { - throw new Error(`Ingestion source ${ingestionSourceId} not found.`); - } + const connector = EmailProviderFactory.createConnector(source); - logger.info(`Starting bulk import for source: ${source.name} (${source.id})`); - await IngestionService.update(ingestionSourceId, { - status: 'importing', - lastSyncStartedAt: new Date() - }); + try { + if (connector.listAllUsers) { + // For multi-mailbox providers, dispatch a job for each user + for await (const user of connector.listAllUsers()) { + const userEmail = user.primaryEmail; + if (userEmail) { + await ingestionQueue.add('process-mailbox', { + ingestionSourceId: source.id, + userEmail: userEmail, + }); + } + } + } else { + // For single-mailbox providers, dispatch a single job + await ingestionQueue.add('process-mailbox', { + ingestionSourceId: source.id, + userEmail: + source.credentials.type === 'generic_imap' + ? source.credentials.username + : 'Default', + }); + } + } catch (error) { + logger.error(`Bulk import failed for source: ${source.name} (${source.id})`, error); + await IngestionService.update(ingestionSourceId, { + status: 'error', + lastSyncFinishedAt: new Date(), + lastSyncStatusMessage: + error instanceof Error ? error.message : 'An unknown error occurred.', + }); + throw error; // Re-throw to allow BullMQ to handle the job failure + } + } - const connector = EmailProviderFactory.createConnector(source); + public async processEmail( + email: EmailObject, + source: IngestionSource, + storage: StorageService, + userEmail: string + ): Promise { + try { + // Generate a unique message ID for the email. If the email already has a message-id header, use that. + // Otherwise, generate a new one based on the email's hash, source ID, and email ID. + const messageIdHeader = email.headers.get('message-id'); + let messageId: string | undefined; + if (Array.isArray(messageIdHeader)) { + messageId = messageIdHeader[0]; + } else if (typeof messageIdHeader === 'string') { + messageId = messageIdHeader; + } + if (!messageId) { + messageId = `generated-${createHash('sha256') + .update(email.eml ?? Buffer.from(email.body, 'utf-8')) + .digest('hex')}-${source.id}-${email.id}`; + } + // Check if an email with the same message ID has already been imported for the current ingestion source. This is to prevent duplicate imports when an email is present in multiple mailboxes (e.g., "Inbox" and "All Mail"). + const existingEmail = await db.query.archivedEmails.findFirst({ + where: and( + eq(archivedEmails.messageIdHeader, messageId), + eq(archivedEmails.ingestionSourceId, source.id) + ), + }); - try { - if (connector.listAllUsers) { - // For multi-mailbox providers, dispatch a job for each user - for await (const user of connector.listAllUsers()) { - const userEmail = user.primaryEmail; - if (userEmail) { - await ingestionQueue.add('process-mailbox', { - ingestionSourceId: source.id, - userEmail: userEmail, - }); - } - } - } else { - // For single-mailbox providers, dispatch a single job - await ingestionQueue.add('process-mailbox', { - ingestionSourceId: source.id, - userEmail: source.credentials.type === 'generic_imap' ? source.credentials.username : 'Default' - }); - } - } catch (error) { - logger.error(`Bulk import failed for source: ${source.name} (${source.id})`, error); - await IngestionService.update(ingestionSourceId, { - status: 'error', - lastSyncFinishedAt: new Date(), - lastSyncStatusMessage: error instanceof Error ? error.message : 'An unknown error occurred.' - }); - throw error; // Re-throw to allow BullMQ to handle the job failure - } - } + if (existingEmail) { + logger.info( + { messageId, ingestionSourceId: source.id }, + 'Skipping duplicate email' + ); + return; + } - public async processEmail( - email: EmailObject, - source: IngestionSource, - storage: StorageService, - userEmail: string - ): Promise { - try { - // Generate a unique message ID for the email. If the email already has a message-id header, use that. - // Otherwise, generate a new one based on the email's hash, source ID, and email ID. - const messageIdHeader = email.headers.get('message-id'); - let messageId: string | undefined; - if (Array.isArray(messageIdHeader)) { - messageId = messageIdHeader[0]; - } else if (typeof messageIdHeader === 'string') { - messageId = messageIdHeader; - } - if (!messageId) { - messageId = `generated-${createHash('sha256').update(email.eml ?? Buffer.from(email.body, 'utf-8')).digest('hex')}-${source.id}-${email.id}`; - } - // Check if an email with the same message ID has already been imported for the current ingestion source. This is to prevent duplicate imports when an email is present in multiple mailboxes (e.g., "Inbox" and "All Mail"). - const existingEmail = await db.query.archivedEmails.findFirst({ - where: and( - eq(archivedEmails.messageIdHeader, messageId), - eq(archivedEmails.ingestionSourceId, source.id) - ) - }); + const emlBuffer = email.eml ?? Buffer.from(email.body, 'utf-8'); + const emailHash = createHash('sha256').update(emlBuffer).digest('hex'); + const sanitizedPath = email.path ? email.path : ''; + const emailPath = `${config.storage.openArchiverFolderName}/${source.name.replaceAll(' ', '-')}-${source.id}/emails/${sanitizedPath}${email.id}.eml`; + await storage.put(emailPath, emlBuffer); - if (existingEmail) { - logger.info({ messageId, ingestionSourceId: source.id }, 'Skipping duplicate email'); - return; - } + const [archivedEmail] = await db + .insert(archivedEmails) + .values({ + ingestionSourceId: source.id, + userEmail, + threadId: email.threadId, + messageIdHeader: messageId, + sentAt: email.receivedAt, + subject: email.subject, + senderName: email.from[0]?.name, + senderEmail: email.from[0]?.address, + recipients: { + to: email.to, + cc: email.cc, + bcc: email.bcc, + }, + storagePath: emailPath, + storageHashSha256: emailHash, + sizeBytes: emlBuffer.length, + hasAttachments: email.attachments.length > 0, + path: email.path, + tags: email.tags, + }) + .returning(); - const emlBuffer = email.eml ?? Buffer.from(email.body, 'utf-8'); - const emailHash = createHash('sha256').update(emlBuffer).digest('hex'); - const sanitizedPath = email.path ? email.path : ''; - const emailPath = `${config.storage.openArchiverFolderName}/${source.name.replaceAll(' ', '-')}-${source.id}/emails/${sanitizedPath}${email.id}.eml`; - await storage.put(emailPath, emlBuffer); + if (email.attachments.length > 0) { + for (const attachment of email.attachments) { + const attachmentBuffer = attachment.content; + const attachmentHash = createHash('sha256') + .update(attachmentBuffer) + .digest('hex'); + const attachmentPath = `${config.storage.openArchiverFolderName}/${source.name.replaceAll(' ', '-')}-${source.id}/attachments/${attachment.filename}`; + await storage.put(attachmentPath, attachmentBuffer); - const [archivedEmail] = await db - .insert(archivedEmails) - .values({ - ingestionSourceId: source.id, - userEmail, - threadId: email.threadId, - messageIdHeader: messageId, - sentAt: email.receivedAt, - subject: email.subject, - senderName: email.from[0]?.name, - senderEmail: email.from[0]?.address, - recipients: { - to: email.to, - cc: email.cc, - bcc: email.bcc - }, - storagePath: emailPath, - storageHashSha256: emailHash, - sizeBytes: emlBuffer.length, - hasAttachments: email.attachments.length > 0, - path: email.path, - tags: email.tags - }) - .returning(); + const [newAttachment] = await db + .insert(attachmentsSchema) + .values({ + filename: attachment.filename, + mimeType: attachment.contentType, + sizeBytes: attachment.size, + contentHashSha256: attachmentHash, + storagePath: attachmentPath, + }) + .onConflictDoUpdate({ + target: attachmentsSchema.contentHashSha256, + set: { filename: attachment.filename }, + }) + .returning(); - if (email.attachments.length > 0) { - for (const attachment of email.attachments) { - const attachmentBuffer = attachment.content; - const attachmentHash = createHash('sha256').update(attachmentBuffer).digest('hex'); - const attachmentPath = `${config.storage.openArchiverFolderName}/${source.name.replaceAll(' ', '-')}-${source.id}/attachments/${attachment.filename}`; - await storage.put(attachmentPath, attachmentBuffer); - - const [newAttachment] = await db - .insert(attachmentsSchema) - .values({ - filename: attachment.filename, - mimeType: attachment.contentType, - sizeBytes: attachment.size, - contentHashSha256: attachmentHash, - storagePath: attachmentPath - }) - .onConflictDoUpdate({ - target: attachmentsSchema.contentHashSha256, - set: { filename: attachment.filename } - }) - .returning(); - - await db - .insert(emailAttachments) - .values({ - emailId: archivedEmail.id, - attachmentId: newAttachment.id - }) - .onConflictDoNothing(); - } - } - // adding to indexing queue - //Instead: index by email (raw email object, ingestion id) - logger.info({ emailId: archivedEmail.id }, 'Indexing email'); - // await indexingQueue.add('index-email', { - // emailId: archivedEmail.id, - // }); - const searchService = new SearchService(); - const storageService = new StorageService(); - const databaseService = new DatabaseService(); - const indexingService = new IndexingService(databaseService, searchService, storageService); - await indexingService.indexByEmail(email, source.id, archivedEmail.id); - } catch (error) { - logger.error({ - message: `Failed to process email ${email.id} for source ${source.id}`, - error, - emailId: email.id, - ingestionSourceId: source.id - }); - } - } + await db + .insert(emailAttachments) + .values({ + emailId: archivedEmail.id, + attachmentId: newAttachment.id, + }) + .onConflictDoNothing(); + } + } + // adding to indexing queue + //Instead: index by email (raw email object, ingestion id) + logger.info({ emailId: archivedEmail.id }, 'Indexing email'); + // await indexingQueue.add('index-email', { + // emailId: archivedEmail.id, + // }); + const searchService = new SearchService(); + const storageService = new StorageService(); + const databaseService = new DatabaseService(); + const indexingService = new IndexingService( + databaseService, + searchService, + storageService + ); + await indexingService.indexByEmail(email, source.id, archivedEmail.id); + } catch (error) { + logger.error({ + message: `Failed to process email ${email.id} for source ${source.id}`, + error, + emailId: email.id, + ingestionSourceId: source.id, + }); + } + } } diff --git a/packages/backend/src/services/SearchService.ts b/packages/backend/src/services/SearchService.ts index 096e6af..a0a0bf5 100644 --- a/packages/backend/src/services/SearchService.ts +++ b/packages/backend/src/services/SearchService.ts @@ -3,116 +3,122 @@ import { config } from '../config'; import type { SearchQuery, SearchResult, EmailDocument, TopSender } from '@open-archiver/types'; export class SearchService { - private client: MeiliSearch; + private client: MeiliSearch; - constructor() { - this.client = new MeiliSearch({ - host: config.search.host, - apiKey: config.search.apiKey, - }); - } + constructor() { + this.client = new MeiliSearch({ + host: config.search.host, + apiKey: config.search.apiKey, + }); + } - public async getIndex>(name: string): Promise> { - return this.client.index(name); - } + public async getIndex>(name: string): Promise> { + return this.client.index(name); + } - public async addDocuments>( - indexName: string, - documents: T[], - primaryKey?: string - ) { - const index = await this.getIndex(indexName); - if (primaryKey) { - index.update({ primaryKey }); - } - return index.addDocuments(documents); - } + public async addDocuments>( + indexName: string, + documents: T[], + primaryKey?: string + ) { + const index = await this.getIndex(indexName); + if (primaryKey) { + index.update({ primaryKey }); + } + return index.addDocuments(documents); + } - public async search>(indexName: string, query: string, options?: any) { - const index = await this.getIndex(indexName); - return index.search(query, options); - } + public async search>( + indexName: string, + query: string, + options?: any + ) { + const index = await this.getIndex(indexName); + return index.search(query, options); + } - public async deleteDocuments(indexName: string, ids: string[]) { - const index = await this.getIndex(indexName); - return index.deleteDocuments(ids); - } + public async deleteDocuments(indexName: string, ids: string[]) { + const index = await this.getIndex(indexName); + return index.deleteDocuments(ids); + } - public async deleteDocumentsByFilter(indexName: string, filter: string | string[]) { - const index = await this.getIndex(indexName); - return index.deleteDocuments({ filter }); - } + public async deleteDocumentsByFilter(indexName: string, filter: string | string[]) { + const index = await this.getIndex(indexName); + return index.deleteDocuments({ filter }); + } - public async searchEmails(dto: SearchQuery): Promise { - const { query, filters, page = 1, limit = 10, matchingStrategy = 'last' } = dto; - const index = await this.getIndex('emails'); + public async searchEmails(dto: SearchQuery): Promise { + const { query, filters, page = 1, limit = 10, matchingStrategy = 'last' } = dto; + const index = await this.getIndex('emails'); - const searchParams: SearchParams = { - limit, - offset: (page - 1) * limit, - attributesToHighlight: ['*'], - showMatchesPosition: true, - sort: ['timestamp:desc'], - matchingStrategy - }; + const searchParams: SearchParams = { + limit, + offset: (page - 1) * limit, + attributesToHighlight: ['*'], + showMatchesPosition: true, + sort: ['timestamp:desc'], + matchingStrategy, + }; - if (filters) { - const filterStrings = Object.entries(filters).map(([key, value]) => { - if (typeof value === 'string') { - return `${key} = '${value}'`; - } - return `${key} = ${value}`; - }); - searchParams.filter = filterStrings.join(' AND '); - } + if (filters) { + const filterStrings = Object.entries(filters).map(([key, value]) => { + if (typeof value === 'string') { + return `${key} = '${value}'`; + } + return `${key} = ${value}`; + }); + searchParams.filter = filterStrings.join(' AND '); + } - const searchResults = await index.search(query, searchParams); + const searchResults = await index.search(query, searchParams); - return { - hits: searchResults.hits, - total: searchResults.estimatedTotalHits ?? searchResults.hits.length, - page, - limit, - totalPages: Math.ceil((searchResults.estimatedTotalHits ?? searchResults.hits.length) / limit), - processingTimeMs: searchResults.processingTimeMs - }; - } + return { + hits: searchResults.hits, + total: searchResults.estimatedTotalHits ?? searchResults.hits.length, + page, + limit, + totalPages: Math.ceil( + (searchResults.estimatedTotalHits ?? searchResults.hits.length) / limit + ), + processingTimeMs: searchResults.processingTimeMs, + }; + } - public async getTopSenders(limit = 10): Promise { - const index = await this.getIndex('emails'); - const searchResults = await index.search('', { - facets: ['from'], - limit: 0 - }); + public async getTopSenders(limit = 10): Promise { + const index = await this.getIndex('emails'); + const searchResults = await index.search('', { + facets: ['from'], + limit: 0, + }); - if (!searchResults.facetDistribution?.from) { - return []; - } + if (!searchResults.facetDistribution?.from) { + return []; + } - // Sort and take top N - const sortedSenders = Object.entries(searchResults.facetDistribution.from) - .sort(([, countA], [, countB]) => countB - countA) - .slice(0, limit) - .map(([sender, count]) => ({ sender, count })); + // Sort and take top N + const sortedSenders = Object.entries(searchResults.facetDistribution.from) + .sort(([, countA], [, countB]) => countB - countA) + .slice(0, limit) + .map(([sender, count]) => ({ sender, count })); - return sortedSenders; - } + return sortedSenders; + } - public async configureEmailIndex() { - const index = await this.getIndex('emails'); - await index.updateSettings({ - searchableAttributes: [ - 'subject', - 'body', - 'from', - 'to', - 'cc', - 'bcc', - 'attachments.filename', - 'attachments.content', - ], - filterableAttributes: ['from', 'to', 'cc', 'bcc', 'timestamp', 'ingestionSourceId'], - sortableAttributes: ['timestamp'] - }); - } + public async configureEmailIndex() { + const index = await this.getIndex('emails'); + await index.updateSettings({ + searchableAttributes: [ + 'subject', + 'body', + 'from', + 'to', + 'cc', + 'bcc', + 'attachments.filename', + 'attachments.content', + ], + filterableAttributes: ['from', 'to', 'cc', 'bcc', 'timestamp', 'ingestionSourceId'], + sortableAttributes: ['timestamp'], + }); + } } diff --git a/packages/backend/src/services/StorageService.ts b/packages/backend/src/services/StorageService.ts index 815d274..a486fe1 100644 --- a/packages/backend/src/services/StorageService.ts +++ b/packages/backend/src/services/StorageService.ts @@ -4,34 +4,34 @@ import { S3StorageProvider } from './storage/S3StorageProvider'; import { config } from '../config/index'; export class StorageService implements IStorageProvider { - private provider: IStorageProvider; + private provider: IStorageProvider; - constructor(storageConfig: StorageConfig = config.storage) { - switch (storageConfig.type) { - case 'local': - this.provider = new LocalFileSystemProvider(storageConfig); - break; - case 's3': - this.provider = new S3StorageProvider(storageConfig); - break; - default: - throw new Error('Invalid storage provider type'); - } - } + constructor(storageConfig: StorageConfig = config.storage) { + switch (storageConfig.type) { + case 'local': + this.provider = new LocalFileSystemProvider(storageConfig); + break; + case 's3': + this.provider = new S3StorageProvider(storageConfig); + break; + default: + throw new Error('Invalid storage provider type'); + } + } - put(path: string, content: Buffer | NodeJS.ReadableStream): Promise { - return this.provider.put(path, content); - } + put(path: string, content: Buffer | NodeJS.ReadableStream): Promise { + return this.provider.put(path, content); + } - get(path: string): Promise { - return this.provider.get(path); - } + get(path: string): Promise { + return this.provider.get(path); + } - delete(path: string): Promise { - return this.provider.delete(path); - } + delete(path: string): Promise { + return this.provider.delete(path); + } - exists(path: string): Promise { - return this.provider.exists(path); - } + exists(path: string): Promise { + return this.provider.exists(path); + } } diff --git a/packages/backend/src/services/UserService.ts b/packages/backend/src/services/UserService.ts index af4ec07..7305478 100644 --- a/packages/backend/src/services/UserService.ts +++ b/packages/backend/src/services/UserService.ts @@ -6,81 +6,95 @@ import type { PolicyStatement, User } from '@open-archiver/types'; import { PolicyValidator } from '../iam-policy/policy-validator'; export class UserService { - /** - * Finds a user by their email address. - * @param email The email address of the user to find. - * @returns The user object if found, otherwise null. - */ - public async findByEmail(email: string): Promise<(typeof schema.users.$inferSelect) | null> { - const user = await db.query.users.findFirst({ - where: eq(schema.users.email, email) - }); - return user || null; - } + /** + * Finds a user by their email address. + * @param email The email address of the user to find. + * @returns The user object if found, otherwise null. + */ + public async findByEmail(email: string): Promise { + const user = await db.query.users.findFirst({ + where: eq(schema.users.email, email), + }); + return user || null; + } - /** - * Finds a user by their ID. - * @param id The ID of the user to find. - * @returns The user object if found, otherwise null. - */ - public async findById(id: string): Promise<(typeof schema.users.$inferSelect) | null> { - const user = await db.query.users.findFirst({ - where: eq(schema.users.id, id) - }); - return user || null; - } + /** + * Finds a user by their ID. + * @param id The ID of the user to find. + * @returns The user object if found, otherwise null. + */ + public async findById(id: string): Promise { + const user = await db.query.users.findFirst({ + where: eq(schema.users.id, id), + }); + return user || null; + } - /** - * Creates an admin user in the database. The user created will be assigned the 'Super Admin' role. - * - * Caution ⚠️: This action can only be allowed in the initial setup - * - * @param userDetails The details of the user to create. - * @param isSetup Is this an initial setup? - * @returns The newly created user object. - */ - public async createAdminUser(userDetails: Pick & { password?: string; }, isSetup: boolean): Promise<(typeof schema.users.$inferSelect)> { - if (!isSetup) { - throw Error('This operation is only allowed upon initial setup.'); - } - const { email, first_name, last_name, password } = userDetails; - const userCountResult = await db.select({ count: sql`count(*)` }).from(schema.users); - const isFirstUser = Number(userCountResult[0].count) === 0; - if (!isFirstUser) { - throw Error('This operation is only allowed upon initial setup.'); - } - const hashedPassword = password ? await hash(password, 10) : undefined; + /** + * Creates an admin user in the database. The user created will be assigned the 'Super Admin' role. + * + * Caution ⚠️: This action can only be allowed in the initial setup + * + * @param userDetails The details of the user to create. + * @param isSetup Is this an initial setup? + * @returns The newly created user object. + */ + public async createAdminUser( + userDetails: Pick & { password?: string }, + isSetup: boolean + ): Promise { + if (!isSetup) { + throw Error('This operation is only allowed upon initial setup.'); + } + const { email, first_name, last_name, password } = userDetails; + const userCountResult = await db + .select({ count: sql`count(*)` }) + .from(schema.users); + const isFirstUser = Number(userCountResult[0].count) === 0; + if (!isFirstUser) { + throw Error('This operation is only allowed upon initial setup.'); + } + const hashedPassword = password ? await hash(password, 10) : undefined; - const newUser = await db.insert(schema.users).values({ - email, - first_name, - last_name, - password: hashedPassword, - }).returning(); + const newUser = await db + .insert(schema.users) + .values({ + email, + first_name, + last_name, + password: hashedPassword, + }) + .returning(); - // find super admin role - let superAdminRole = await db.query.roles.findFirst({ - where: eq(schema.roles.name, 'Super Admin') - }); + // find super admin role + let superAdminRole = await db.query.roles.findFirst({ + where: eq(schema.roles.name, 'Super Admin'), + }); - if (!superAdminRole) { - const suerAdminPolicies: PolicyStatement[] = [{ - Effect: 'Allow', - Action: ['*'], - Resource: ['*'] - }]; - superAdminRole = (await db.insert(schema.roles).values({ - name: 'Super Admin', - policies: suerAdminPolicies - }).returning())[0]; - } + if (!superAdminRole) { + const suerAdminPolicies: PolicyStatement[] = [ + { + Effect: 'Allow', + Action: ['*'], + Resource: ['*'], + }, + ]; + superAdminRole = ( + await db + .insert(schema.roles) + .values({ + name: 'Super Admin', + policies: suerAdminPolicies, + }) + .returning() + )[0]; + } - await db.insert(schema.userRoles).values({ - userId: newUser[0].id, - roleId: superAdminRole.id - }); + await db.insert(schema.userRoles).values({ + userId: newUser[0].id, + roleId: superAdminRole.id, + }); - - return newUser[0]; - } + return newUser[0]; + } } diff --git a/packages/backend/src/services/ingestion-connectors/EMLConnector.ts b/packages/backend/src/services/ingestion-connectors/EMLConnector.ts index 51bb255..e985aec 100644 --- a/packages/backend/src/services/ingestion-connectors/EMLConnector.ts +++ b/packages/backend/src/services/ingestion-connectors/EMLConnector.ts @@ -1,4 +1,10 @@ -import type { EMLImportCredentials, EmailObject, EmailAddress, SyncState, MailboxUser } from '@open-archiver/types'; +import type { + EMLImportCredentials, + EmailObject, + EmailAddress, + SyncState, + MailboxUser, +} from '@open-archiver/types'; import type { IEmailConnector } from '../EmailProviderFactory'; import { simpleParser, ParsedMail, Attachment, AddressObject } from 'mailparser'; import { logger } from '../../config/logger'; @@ -11,194 +17,209 @@ import { createReadStream, promises as fs, createWriteStream } from 'fs'; import * as yauzl from 'yauzl'; const streamToBuffer = (stream: Readable): Promise => { - return new Promise((resolve, reject) => { - const chunks: Buffer[] = []; - stream.on('data', (chunk) => chunks.push(chunk)); - stream.on('error', reject); - stream.on('end', () => resolve(Buffer.concat(chunks))); - }); + return new Promise((resolve, reject) => { + const chunks: Buffer[] = []; + stream.on('data', (chunk) => chunks.push(chunk)); + stream.on('error', reject); + stream.on('end', () => resolve(Buffer.concat(chunks))); + }); }; export class EMLConnector implements IEmailConnector { - private storage: StorageService; + private storage: StorageService; - constructor(private credentials: EMLImportCredentials) { - this.storage = new StorageService(); - } + constructor(private credentials: EMLImportCredentials) { + this.storage = new StorageService(); + } - public async testConnection(): Promise { - try { - if (!this.credentials.uploadedFilePath) { - throw Error("EML file path not provided."); - } - if (!this.credentials.uploadedFilePath.includes('.zip')) { - throw Error("Provided file is not in the ZIP format."); - } - const fileExist = await this.storage.exists(this.credentials.uploadedFilePath); - if (!fileExist) { - throw Error("EML file upload not finished yet, please wait."); - } + public async testConnection(): Promise { + try { + if (!this.credentials.uploadedFilePath) { + throw Error('EML file path not provided.'); + } + if (!this.credentials.uploadedFilePath.includes('.zip')) { + throw Error('Provided file is not in the ZIP format.'); + } + const fileExist = await this.storage.exists(this.credentials.uploadedFilePath); + if (!fileExist) { + throw Error('EML file upload not finished yet, please wait.'); + } - return true; - } catch (error) { - logger.error({ error, credentials: this.credentials }, 'EML file validation failed.'); - throw error; - } - } + return true; + } catch (error) { + logger.error({ error, credentials: this.credentials }, 'EML file validation failed.'); + throw error; + } + } - public async *listAllUsers(): AsyncGenerator { - const displayName = this.credentials.uploadedFileName || `eml-import-${new Date().getTime()}`; - logger.info(`Found potential mailbox: ${displayName}`); - const constructedPrimaryEmail = `${displayName.replace(/ /g, '.').toLowerCase()}@eml.local`; - yield { - id: constructedPrimaryEmail, - primaryEmail: constructedPrimaryEmail, - displayName: displayName, - }; - } + public async *listAllUsers(): AsyncGenerator { + const displayName = + this.credentials.uploadedFileName || `eml-import-${new Date().getTime()}`; + logger.info(`Found potential mailbox: ${displayName}`); + const constructedPrimaryEmail = `${displayName.replace(/ /g, '.').toLowerCase()}@eml.local`; + yield { + id: constructedPrimaryEmail, + primaryEmail: constructedPrimaryEmail, + displayName: displayName, + }; + } - public async *fetchEmails(userEmail: string, syncState?: SyncState | null): AsyncGenerator { - const fileStream = await this.storage.get(this.credentials.uploadedFilePath); - const tempDir = await fs.mkdtemp(join('/tmp', 'eml-import-')); - const unzippedPath = join(tempDir, 'unzipped'); - await fs.mkdir(unzippedPath); - const zipFilePath = join(tempDir, 'eml.zip'); + public async *fetchEmails( + userEmail: string, + syncState?: SyncState | null + ): AsyncGenerator { + const fileStream = await this.storage.get(this.credentials.uploadedFilePath); + const tempDir = await fs.mkdtemp(join('/tmp', 'eml-import-')); + const unzippedPath = join(tempDir, 'unzipped'); + await fs.mkdir(unzippedPath); + const zipFilePath = join(tempDir, 'eml.zip'); - try { - await new Promise((resolve, reject) => { - const dest = createWriteStream(zipFilePath); - (fileStream as Readable).pipe(dest); - dest.on('finish', () => resolve()); - dest.on('error', reject); - }); + try { + await new Promise((resolve, reject) => { + const dest = createWriteStream(zipFilePath); + (fileStream as Readable).pipe(dest); + dest.on('finish', () => resolve()); + dest.on('error', reject); + }); - await this.extract(zipFilePath, unzippedPath); + await this.extract(zipFilePath, unzippedPath); - const files = await this.getAllFiles(unzippedPath); + const files = await this.getAllFiles(unzippedPath); - for (const file of files) { - if (file.endsWith('.eml')) { - try { - // logger.info({ file }, 'Processing EML file.'); - const stream = createReadStream(file); - const content = await streamToBuffer(stream); - // logger.info({ file, size: content.length }, 'Read file to buffer.'); - let relativePath = file.substring(unzippedPath.length + 1); - if (dirname(relativePath) === '.') { - relativePath = ''; - } else { - relativePath = dirname(relativePath); - } - const emailObject = await this.parseMessage(content, relativePath); - // logger.info({ file, messageId: emailObject.id }, 'Parsed email message.'); - yield emailObject; - } catch (error) { - logger.error({ error, file }, 'Failed to process a single EML file. Skipping.'); - } - } - } - } catch (error) { - logger.error({ error }, 'Failed to fetch email.'); - throw error; - } finally { - await fs.rm(tempDir, { recursive: true, force: true }); - } - } + for (const file of files) { + if (file.endsWith('.eml')) { + try { + // logger.info({ file }, 'Processing EML file.'); + const stream = createReadStream(file); + const content = await streamToBuffer(stream); + // logger.info({ file, size: content.length }, 'Read file to buffer.'); + let relativePath = file.substring(unzippedPath.length + 1); + if (dirname(relativePath) === '.') { + relativePath = ''; + } else { + relativePath = dirname(relativePath); + } + const emailObject = await this.parseMessage(content, relativePath); + // logger.info({ file, messageId: emailObject.id }, 'Parsed email message.'); + yield emailObject; + } catch (error) { + logger.error( + { error, file }, + 'Failed to process a single EML file. Skipping.' + ); + } + } + } + } catch (error) { + logger.error({ error }, 'Failed to fetch email.'); + throw error; + } finally { + await fs.rm(tempDir, { recursive: true, force: true }); + } + } - private extract(zipFilePath: string, dest: string): Promise { - return new Promise((resolve, reject) => { - yauzl.open(zipFilePath, { lazyEntries: true, decodeStrings: false }, (err, zipfile) => { - if (err) reject(err); - zipfile.on('error', reject); - zipfile.readEntry(); - zipfile.on('entry', (entry) => { - const fileName = entry.fileName.toString('utf8'); - // Ignore macOS-specific metadata files. - if (fileName.startsWith('__MACOSX/')) { - zipfile.readEntry(); - return; - } - const entryPath = join(dest, fileName); - if (/\/$/.test(fileName)) { - fs.mkdir(entryPath, { recursive: true }).then(() => zipfile.readEntry()).catch(reject); - } else { - zipfile.openReadStream(entry, (err, readStream) => { - if (err) reject(err); - const writeStream = createWriteStream(entryPath); - readStream.pipe(writeStream); - writeStream.on('finish', () => zipfile.readEntry()); - writeStream.on('error', reject); - }); - } - }); - zipfile.on('end', () => resolve()); - }); - }); - } + private extract(zipFilePath: string, dest: string): Promise { + return new Promise((resolve, reject) => { + yauzl.open(zipFilePath, { lazyEntries: true, decodeStrings: false }, (err, zipfile) => { + if (err) reject(err); + zipfile.on('error', reject); + zipfile.readEntry(); + zipfile.on('entry', (entry) => { + const fileName = entry.fileName.toString('utf8'); + // Ignore macOS-specific metadata files. + if (fileName.startsWith('__MACOSX/')) { + zipfile.readEntry(); + return; + } + const entryPath = join(dest, fileName); + if (/\/$/.test(fileName)) { + fs.mkdir(entryPath, { recursive: true }) + .then(() => zipfile.readEntry()) + .catch(reject); + } else { + zipfile.openReadStream(entry, (err, readStream) => { + if (err) reject(err); + const writeStream = createWriteStream(entryPath); + readStream.pipe(writeStream); + writeStream.on('finish', () => zipfile.readEntry()); + writeStream.on('error', reject); + }); + } + }); + zipfile.on('end', () => resolve()); + }); + }); + } - private async getAllFiles(dirPath: string, arrayOfFiles: string[] = []): Promise { - const files = await fs.readdir(dirPath); + private async getAllFiles(dirPath: string, arrayOfFiles: string[] = []): Promise { + const files = await fs.readdir(dirPath); - for (const file of files) { - const fullPath = join(dirPath, file); - if ((await fs.stat(fullPath)).isDirectory()) { - await this.getAllFiles(fullPath, arrayOfFiles); - } else { - arrayOfFiles.push(fullPath); - } - } + for (const file of files) { + const fullPath = join(dirPath, file); + if ((await fs.stat(fullPath)).isDirectory()) { + await this.getAllFiles(fullPath, arrayOfFiles); + } else { + arrayOfFiles.push(fullPath); + } + } - return arrayOfFiles; - } + return arrayOfFiles; + } - private async parseMessage(emlBuffer: Buffer, path: string): Promise { - const parsedEmail: ParsedMail = await simpleParser(emlBuffer); + private async parseMessage(emlBuffer: Buffer, path: string): Promise { + const parsedEmail: ParsedMail = await simpleParser(emlBuffer); - const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ - filename: attachment.filename || 'untitled', - contentType: attachment.contentType, - size: attachment.size, - content: attachment.content as Buffer - })); + const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ + filename: attachment.filename || 'untitled', + contentType: attachment.contentType, + size: attachment.size, + content: attachment.content as Buffer, + })); - const mapAddresses = (addresses: AddressObject | AddressObject[] | undefined): EmailAddress[] => { - if (!addresses) return []; - const addressArray = Array.isArray(addresses) ? addresses : [addresses]; - return addressArray.flatMap(a => a.value.map(v => ({ name: v.name, address: v.address?.replaceAll(`'`, '') || '' }))); - }; + const mapAddresses = ( + addresses: AddressObject | AddressObject[] | undefined + ): EmailAddress[] => { + if (!addresses) return []; + const addressArray = Array.isArray(addresses) ? addresses : [addresses]; + return addressArray.flatMap((a) => + a.value.map((v) => ({ + name: v.name, + address: v.address?.replaceAll(`'`, '') || '', + })) + ); + }; - const threadId = getThreadId(parsedEmail.headers); - let messageId = parsedEmail.messageId; + const threadId = getThreadId(parsedEmail.headers); + let messageId = parsedEmail.messageId; - if (!messageId) { - messageId = `generated-${createHash('sha256').update(emlBuffer).digest('hex')}`; - } + if (!messageId) { + messageId = `generated-${createHash('sha256').update(emlBuffer).digest('hex')}`; + } - const from = mapAddresses(parsedEmail.from); - if (from.length === 0) { - from.push({ name: 'No Sender', address: 'No Sender' }); - } + const from = mapAddresses(parsedEmail.from); + if (from.length === 0) { + from.push({ name: 'No Sender', address: 'No Sender' }); + } + return { + id: messageId, + threadId: threadId, + from, + to: mapAddresses(parsedEmail.to), + cc: mapAddresses(parsedEmail.cc), + bcc: mapAddresses(parsedEmail.bcc), + subject: parsedEmail.subject || '', + body: parsedEmail.text || '', + html: parsedEmail.html || '', + headers: parsedEmail.headers, + attachments, + receivedAt: parsedEmail.date || new Date(), + eml: emlBuffer, + path, + }; + } - return { - id: messageId, - threadId: threadId, - from, - to: mapAddresses(parsedEmail.to), - cc: mapAddresses(parsedEmail.cc), - bcc: mapAddresses(parsedEmail.bcc), - subject: parsedEmail.subject || '', - body: parsedEmail.text || '', - html: parsedEmail.html || '', - headers: parsedEmail.headers, - attachments, - receivedAt: parsedEmail.date || new Date(), - eml: emlBuffer, - path - }; - } - - public getUpdatedSyncState(): SyncState { - return {}; - } + public getUpdatedSyncState(): SyncState { + return {}; + } } diff --git a/packages/backend/src/services/ingestion-connectors/GoogleWorkspaceConnector.ts b/packages/backend/src/services/ingestion-connectors/GoogleWorkspaceConnector.ts index 013d620..e96bd6d 100644 --- a/packages/backend/src/services/ingestion-connectors/GoogleWorkspaceConnector.ts +++ b/packages/backend/src/services/ingestion-connectors/GoogleWorkspaceConnector.ts @@ -1,11 +1,11 @@ import { google } from 'googleapis'; import type { admin_directory_v1, gmail_v1, Common } from 'googleapis'; import type { - GoogleWorkspaceCredentials, - EmailObject, - EmailAddress, - SyncState, - MailboxUser + GoogleWorkspaceCredentials, + EmailObject, + EmailAddress, + SyncState, + MailboxUser, } from '@open-archiver/types'; import type { IEmailConnector } from '../EmailProviderFactory'; import { logger } from '../../config/logger'; @@ -17,347 +17,388 @@ import { getThreadId } from './helpers/utils'; * to access user data on behalf of users in the domain. */ export class GoogleWorkspaceConnector implements IEmailConnector { - private credentials: GoogleWorkspaceCredentials; - private serviceAccountCreds: { client_email: string; private_key: string; }; - private newHistoryId: string | undefined; + private credentials: GoogleWorkspaceCredentials; + private serviceAccountCreds: { client_email: string; private_key: string }; + private newHistoryId: string | undefined; - constructor(credentials: GoogleWorkspaceCredentials) { - this.credentials = credentials; - try { - // Pre-parse the JSON key to catch errors early. - const parsedKey = JSON.parse(this.credentials.serviceAccountKeyJson); - if (!parsedKey.client_email || !parsedKey.private_key) { - throw new Error('Service account key JSON is missing required fields.'); - } - this.serviceAccountCreds = { - client_email: parsedKey.client_email, - private_key: parsedKey.private_key - }; - } catch (error) { - logger.error({ err: error }, 'Failed to parse Google Service Account JSON'); - throw new Error('Invalid Google Service Account JSON key.'); - } - } + constructor(credentials: GoogleWorkspaceCredentials) { + this.credentials = credentials; + try { + // Pre-parse the JSON key to catch errors early. + const parsedKey = JSON.parse(this.credentials.serviceAccountKeyJson); + if (!parsedKey.client_email || !parsedKey.private_key) { + throw new Error('Service account key JSON is missing required fields.'); + } + this.serviceAccountCreds = { + client_email: parsedKey.client_email, + private_key: parsedKey.private_key, + }; + } catch (error) { + logger.error({ err: error }, 'Failed to parse Google Service Account JSON'); + throw new Error('Invalid Google Service Account JSON key.'); + } + } - /** - * Creates an authenticated JWT client capable of impersonating a user. - * @param subject The email address of the user to impersonate. - * @param scopes The OAuth scopes required for the API calls. - * @returns An authenticated JWT client. - */ - private getAuthClient(subject: string, scopes: string[]) { - const jwtClient = new google.auth.JWT({ - email: this.serviceAccountCreds.client_email, - key: this.serviceAccountCreds.private_key, - scopes, - subject - }); - return jwtClient; - } + /** + * Creates an authenticated JWT client capable of impersonating a user. + * @param subject The email address of the user to impersonate. + * @param scopes The OAuth scopes required for the API calls. + * @returns An authenticated JWT client. + */ + private getAuthClient(subject: string, scopes: string[]) { + const jwtClient = new google.auth.JWT({ + email: this.serviceAccountCreds.client_email, + key: this.serviceAccountCreds.private_key, + scopes, + subject, + }); + return jwtClient; + } + /** + * Tests the connection and authentication by attempting to list the first user + * from the directory, impersonating the admin user. + */ + public async testConnection(): Promise { + try { + const authClient = this.getAuthClient(this.credentials.impersonatedAdminEmail, [ + 'https://www.googleapis.com/auth/admin.directory.user.readonly', + ]); - /** - * Tests the connection and authentication by attempting to list the first user - * from the directory, impersonating the admin user. - */ - public async testConnection(): Promise { - try { - const authClient = this.getAuthClient(this.credentials.impersonatedAdminEmail, [ - 'https://www.googleapis.com/auth/admin.directory.user.readonly' - ]); + const admin = google.admin({ + version: 'directory_v1', + auth: authClient, + }); - const admin = google.admin({ - version: 'directory_v1', - auth: authClient - }); + // Perform a simple, low-impact read operation to verify credentials. + await admin.users.list({ + customer: 'my_customer', + maxResults: 1, + orderBy: 'email', + }); - // Perform a simple, low-impact read operation to verify credentials. - await admin.users.list({ - customer: 'my_customer', - maxResults: 1, - orderBy: 'email' - }); + logger.info('Google Workspace connection test successful.'); + return true; + } catch (error) { + logger.error({ err: error }, 'Failed to verify Google Workspace connection'); + throw error; + } + } - logger.info('Google Workspace connection test successful.'); - return true; - } catch (error) { - logger.error({ err: error }, 'Failed to verify Google Workspace connection'); - throw error; - } - } + /** + * Lists all users in the Google Workspace domain. + * This method handles pagination to retrieve the complete list of users. + * @returns An async generator that yields each user object. + */ + public async *listAllUsers(): AsyncGenerator { + const authClient = this.getAuthClient(this.credentials.impersonatedAdminEmail, [ + 'https://www.googleapis.com/auth/admin.directory.user.readonly', + ]); - /** - * Lists all users in the Google Workspace domain. - * This method handles pagination to retrieve the complete list of users. - * @returns An async generator that yields each user object. - */ - public async *listAllUsers(): AsyncGenerator { - const authClient = this.getAuthClient(this.credentials.impersonatedAdminEmail, [ - 'https://www.googleapis.com/auth/admin.directory.user.readonly' - ]); + const admin = google.admin({ version: 'directory_v1', auth: authClient }); + let pageToken: string | undefined = undefined; - const admin = google.admin({ version: 'directory_v1', auth: authClient }); - let pageToken: string | undefined = undefined; + do { + const res: Common.GaxiosResponseWithHTTP2 = + await admin.users.list({ + customer: 'my_customer', + maxResults: 500, // Max allowed per page + pageToken: pageToken, + orderBy: 'email', + }); - do { - const res: Common.GaxiosResponseWithHTTP2 = await admin.users.list({ - customer: 'my_customer', - maxResults: 500, // Max allowed per page - pageToken: pageToken, - orderBy: 'email' - }); + const users = res.data.users; + if (users) { + for (const user of users) { + if (user.id && user.primaryEmail && user.name?.fullName) { + yield { + id: user.id, + primaryEmail: user.primaryEmail, + displayName: user.name.fullName, + }; + } + } + } + pageToken = res.data.nextPageToken ?? undefined; + } while (pageToken); + } - const users = res.data.users; - if (users) { - for (const user of users) { - if (user.id && user.primaryEmail && user.name?.fullName) { - yield { - id: user.id, - primaryEmail: user.primaryEmail, - displayName: user.name.fullName - }; - } - } - } - pageToken = res.data.nextPageToken ?? undefined; - } while (pageToken); - } + /** + * Fetches emails for a single user, starting from a specific history ID. + * This is ideal for continuous synchronization jobs. + * @param userEmail The email of the user whose mailbox will be read. + * @param syncState Optional state containing the startHistoryId. + * @returns An async generator that yields each raw email object. + */ + public async *fetchEmails( + userEmail: string, + syncState?: SyncState | null + ): AsyncGenerator { + const authClient = this.getAuthClient(userEmail, [ + 'https://www.googleapis.com/auth/gmail.readonly', + ]); + const gmail = google.gmail({ version: 'v1', auth: authClient }); + let pageToken: string | undefined = undefined; - /** - * Fetches emails for a single user, starting from a specific history ID. - * This is ideal for continuous synchronization jobs. - * @param userEmail The email of the user whose mailbox will be read. - * @param syncState Optional state containing the startHistoryId. - * @returns An async generator that yields each raw email object. - */ - public async *fetchEmails( - userEmail: string, - syncState?: SyncState | null - ): AsyncGenerator { - const authClient = this.getAuthClient(userEmail, [ - 'https://www.googleapis.com/auth/gmail.readonly' - ]); - const gmail = google.gmail({ version: 'v1', auth: authClient }); - let pageToken: string | undefined = undefined; + const startHistoryId = syncState?.google?.[userEmail]?.historyId; - const startHistoryId = syncState?.google?.[userEmail]?.historyId; + // If no sync state is provided for this user, this is an initial import. Get all messages. + if (!startHistoryId) { + yield* this.fetchAllMessagesForUser(gmail, userEmail); + return; + } - // If no sync state is provided for this user, this is an initial import. Get all messages. - if (!startHistoryId) { - yield* this.fetchAllMessagesForUser(gmail, userEmail); - return; - } + this.newHistoryId = startHistoryId; - this.newHistoryId = startHistoryId; + do { + const historyResponse: Common.GaxiosResponseWithHTTP2 = + await gmail.users.history.list({ + userId: userEmail, + startHistoryId: this.newHistoryId, + pageToken: pageToken, + historyTypes: ['messageAdded'], + }); - do { - const historyResponse: Common.GaxiosResponseWithHTTP2 = await gmail.users.history.list({ - userId: userEmail, - startHistoryId: this.newHistoryId, - pageToken: pageToken, - historyTypes: ['messageAdded'] - }); + const histories = historyResponse.data.history; + if (!histories || histories.length === 0) { + return; + } - const histories = historyResponse.data.history; - if (!histories || histories.length === 0) { - return; - } + for (const historyRecord of histories) { + if (historyRecord.messagesAdded) { + for (const messageAdded of historyRecord.messagesAdded) { + if (messageAdded.message?.id) { + try { + const messageId = messageAdded.message.id; + const metadataResponse = await gmail.users.messages.get({ + userId: userEmail, + id: messageId, + format: 'METADATA', + fields: 'labelIds', + }); + const labels = await this.getLabelDetails( + gmail, + userEmail, + metadataResponse.data.labelIds || [] + ); - for (const historyRecord of histories) { - if (historyRecord.messagesAdded) { - for (const messageAdded of historyRecord.messagesAdded) { - if (messageAdded.message?.id) { - try { - const messageId = messageAdded.message.id; - const metadataResponse = await gmail.users.messages.get({ - userId: userEmail, - id: messageId, - format: 'METADATA', - fields: 'labelIds' - }); - const labels = await this.getLabelDetails(gmail, userEmail, metadataResponse.data.labelIds || []); + const msgResponse = await gmail.users.messages.get({ + userId: userEmail, + id: messageId, + format: 'RAW', + }); - const msgResponse = await gmail.users.messages.get({ - userId: userEmail, - id: messageId, - format: 'RAW' - }); + if (msgResponse.data.raw) { + const rawEmail = Buffer.from(msgResponse.data.raw, 'base64url'); + const parsedEmail: ParsedMail = await simpleParser(rawEmail); + const attachments = parsedEmail.attachments.map( + (attachment: Attachment) => ({ + filename: attachment.filename || 'untitled', + contentType: attachment.contentType, + size: attachment.size, + content: attachment.content as Buffer, + }) + ); + const mapAddresses = ( + addresses: AddressObject | AddressObject[] | undefined + ): EmailAddress[] => { + if (!addresses) return []; + const addressArray = Array.isArray(addresses) + ? addresses + : [addresses]; + return addressArray.flatMap((a) => + a.value.map((v) => ({ + name: v.name, + address: v.address || '', + })) + ); + }; + const threadId = getThreadId(parsedEmail.headers); + console.log('threadId', threadId); + yield { + id: msgResponse.data.id!, + threadId, + userEmail: userEmail, + eml: rawEmail, + from: mapAddresses(parsedEmail.from), + to: mapAddresses(parsedEmail.to), + cc: mapAddresses(parsedEmail.cc), + bcc: mapAddresses(parsedEmail.bcc), + subject: parsedEmail.subject || '', + body: parsedEmail.text || '', + html: parsedEmail.html || '', + headers: parsedEmail.headers, + attachments, + receivedAt: parsedEmail.date || new Date(), + path: labels.path, + tags: labels.tags, + }; + } + } catch (error: any) { + if (error.code === 404) { + logger.warn( + { messageId: messageAdded.message.id, userEmail }, + 'Message not found, skipping.' + ); + } else { + throw error; + } + } + } + } + } + } - if (msgResponse.data.raw) { - const rawEmail = Buffer.from(msgResponse.data.raw, 'base64url'); - const parsedEmail: ParsedMail = await simpleParser(rawEmail); - const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ - filename: attachment.filename || 'untitled', - contentType: attachment.contentType, - size: attachment.size, - content: attachment.content as Buffer - })); - const mapAddresses = (addresses: AddressObject | AddressObject[] | undefined): EmailAddress[] => { - if (!addresses) return []; - const addressArray = Array.isArray(addresses) ? addresses : [addresses]; - return addressArray.flatMap(a => a.value.map(v => ({ name: v.name, address: v.address || '' }))); - }; - const threadId = getThreadId(parsedEmail.headers); - console.log('threadId', threadId); - yield { - id: msgResponse.data.id!, - threadId, - userEmail: userEmail, - eml: rawEmail, - from: mapAddresses(parsedEmail.from), - to: mapAddresses(parsedEmail.to), - cc: mapAddresses(parsedEmail.cc), - bcc: mapAddresses(parsedEmail.bcc), - subject: parsedEmail.subject || '', - body: parsedEmail.text || '', - html: parsedEmail.html || '', - headers: parsedEmail.headers, - attachments, - receivedAt: parsedEmail.date || new Date(), - path: labels.path, - tags: labels.tags - }; - } - } catch (error: any) { - if (error.code === 404) { - logger.warn({ messageId: messageAdded.message.id, userEmail }, 'Message not found, skipping.'); - } else { - throw error; - } - } - } - } - } - } + pageToken = historyResponse.data.nextPageToken ?? undefined; + if (historyResponse.data.historyId) { + this.newHistoryId = historyResponse.data.historyId; + } + } while (pageToken); + } - pageToken = historyResponse.data.nextPageToken ?? undefined; - if (historyResponse.data.historyId) { - this.newHistoryId = historyResponse.data.historyId; - } + private async *fetchAllMessagesForUser( + gmail: gmail_v1.Gmail, + userEmail: string + ): AsyncGenerator { + let pageToken: string | undefined = undefined; + do { + const listResponse: Common.GaxiosResponseWithHTTP2 = + await gmail.users.messages.list({ + userId: userEmail, + pageToken: pageToken, + }); - } while (pageToken); - } + const messages = listResponse.data.messages; + if (!messages || messages.length === 0) { + return; + } - private async *fetchAllMessagesForUser(gmail: gmail_v1.Gmail, userEmail: string): AsyncGenerator { - let pageToken: string | undefined = undefined; - do { - const listResponse: Common.GaxiosResponseWithHTTP2 = await gmail.users.messages.list({ - userId: userEmail, - pageToken: pageToken - }); + for (const message of messages) { + if (message.id) { + try { + const messageId = message.id; + const metadataResponse = await gmail.users.messages.get({ + userId: userEmail, + id: messageId, + format: 'METADATA', + fields: 'labelIds', + }); + const labels = await this.getLabelDetails( + gmail, + userEmail, + metadataResponse.data.labelIds || [] + ); - const messages = listResponse.data.messages; - if (!messages || messages.length === 0) { - return; - } + const msgResponse = await gmail.users.messages.get({ + userId: userEmail, + id: messageId, + format: 'RAW', + }); - for (const message of messages) { - if (message.id) { - try { - const messageId = message.id; - const metadataResponse = await gmail.users.messages.get({ - userId: userEmail, - id: messageId, - format: 'METADATA', - fields: 'labelIds' - }); - const labels = await this.getLabelDetails(gmail, userEmail, metadataResponse.data.labelIds || []); + if (msgResponse.data.raw) { + const rawEmail = Buffer.from(msgResponse.data.raw, 'base64url'); + const parsedEmail: ParsedMail = await simpleParser(rawEmail); + const attachments = parsedEmail.attachments.map( + (attachment: Attachment) => ({ + filename: attachment.filename || 'untitled', + contentType: attachment.contentType, + size: attachment.size, + content: attachment.content as Buffer, + }) + ); + const mapAddresses = ( + addresses: AddressObject | AddressObject[] | undefined + ): EmailAddress[] => { + if (!addresses) return []; + const addressArray = Array.isArray(addresses) + ? addresses + : [addresses]; + return addressArray.flatMap((a) => + a.value.map((v) => ({ name: v.name, address: v.address || '' })) + ); + }; + const threadId = getThreadId(parsedEmail.headers); + console.log('threadId', threadId); + yield { + id: msgResponse.data.id!, + threadId, + userEmail: userEmail, + eml: rawEmail, + from: mapAddresses(parsedEmail.from), + to: mapAddresses(parsedEmail.to), + cc: mapAddresses(parsedEmail.cc), + bcc: mapAddresses(parsedEmail.bcc), + subject: parsedEmail.subject || '', + body: parsedEmail.text || '', + html: parsedEmail.html || '', + headers: parsedEmail.headers, + attachments, + receivedAt: parsedEmail.date || new Date(), + path: labels.path, + tags: labels.tags, + }; + } + } catch (error: any) { + if (error.code === 404) { + logger.warn( + { messageId: message.id, userEmail }, + 'Message not found during initial import, skipping.' + ); + } else { + throw error; + } + } + } + } + pageToken = listResponse.data.nextPageToken ?? undefined; + } while (pageToken); - const msgResponse = await gmail.users.messages.get({ - userId: userEmail, - id: messageId, - format: 'RAW' - }); + // After fetching all messages, get the latest history ID to use as the starting point for the next sync. + const profileResponse = await gmail.users.getProfile({ userId: userEmail }); + if (profileResponse.data.historyId) { + this.newHistoryId = profileResponse.data.historyId; + } + } - if (msgResponse.data.raw) { - const rawEmail = Buffer.from(msgResponse.data.raw, 'base64url'); - const parsedEmail: ParsedMail = await simpleParser(rawEmail); - const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ - filename: attachment.filename || 'untitled', - contentType: attachment.contentType, - size: attachment.size, - content: attachment.content as Buffer - })); - const mapAddresses = (addresses: AddressObject | AddressObject[] | undefined): EmailAddress[] => { - if (!addresses) return []; - const addressArray = Array.isArray(addresses) ? addresses : [addresses]; - return addressArray.flatMap(a => a.value.map(v => ({ name: v.name, address: v.address || '' }))); - }; - const threadId = getThreadId(parsedEmail.headers); - console.log('threadId', threadId); - yield { - id: msgResponse.data.id!, - threadId, - userEmail: userEmail, - eml: rawEmail, - from: mapAddresses(parsedEmail.from), - to: mapAddresses(parsedEmail.to), - cc: mapAddresses(parsedEmail.cc), - bcc: mapAddresses(parsedEmail.bcc), - subject: parsedEmail.subject || '', - body: parsedEmail.text || '', - html: parsedEmail.html || '', - headers: parsedEmail.headers, - attachments, - receivedAt: parsedEmail.date || new Date(), - path: labels.path, - tags: labels.tags - }; - } - } catch (error: any) { - if (error.code === 404) { - logger.warn({ messageId: message.id, userEmail }, 'Message not found during initial import, skipping.'); - } else { - throw error; - } - } - } - } - pageToken = listResponse.data.nextPageToken ?? undefined; - } while (pageToken); + public getUpdatedSyncState(userEmail: string): SyncState { + if (!this.newHistoryId) { + return {}; + } + return { + google: { + [userEmail]: { + historyId: this.newHistoryId, + }, + }, + }; + } - // After fetching all messages, get the latest history ID to use as the starting point for the next sync. - const profileResponse = await gmail.users.getProfile({ userId: userEmail }); - if (profileResponse.data.historyId) { - this.newHistoryId = profileResponse.data.historyId; - } - } + private labelCache: Map = new Map(); - public getUpdatedSyncState(userEmail: string): SyncState { - if (!this.newHistoryId) { - return {}; - } - return { - google: { - [userEmail]: { - historyId: this.newHistoryId - } - } - }; - } + private async getLabelDetails( + gmail: gmail_v1.Gmail, + userEmail: string, + labelIds: string[] + ): Promise<{ path: string; tags: string[] }> { + const tags: string[] = []; + let path = ''; - private labelCache: Map = new Map(); + for (const labelId of labelIds) { + let label = this.labelCache.get(labelId); + if (!label) { + const res = await gmail.users.labels.get({ userId: userEmail, id: labelId }); + label = res.data; + this.labelCache.set(labelId, label); + } - private async getLabelDetails(gmail: gmail_v1.Gmail, userEmail: string, labelIds: string[]): Promise<{ path: string, tags: string[]; }> { - const tags: string[] = []; - let path = ''; + if (label.name) { + tags.push(label.name); + if (label.type === 'user') { + path = path ? `${path}/${label.name}` : label.name; + } + } + } - for (const labelId of labelIds) { - let label = this.labelCache.get(labelId); - if (!label) { - const res = await gmail.users.labels.get({ userId: userEmail, id: labelId }); - label = res.data; - this.labelCache.set(labelId, label); - } - - if (label.name) { - tags.push(label.name); - if (label.type === 'user') { - path = path ? `${path}/${label.name}` : label.name; - } - } - } - - return { path, tags }; - } + return { path, tags }; + } } diff --git a/packages/backend/src/services/ingestion-connectors/ImapConnector.ts b/packages/backend/src/services/ingestion-connectors/ImapConnector.ts index 6e3acfa..3c3649e 100644 --- a/packages/backend/src/services/ingestion-connectors/ImapConnector.ts +++ b/packages/backend/src/services/ingestion-connectors/ImapConnector.ts @@ -1,4 +1,10 @@ -import type { GenericImapCredentials, EmailObject, EmailAddress, SyncState, MailboxUser } from '@open-archiver/types'; +import type { + GenericImapCredentials, + EmailObject, + EmailAddress, + SyncState, + MailboxUser, +} from '@open-archiver/types'; import type { IEmailConnector } from '../EmailProviderFactory'; import { ImapFlow } from 'imapflow'; import { simpleParser, ParsedMail, Attachment, AddressObject, Headers } from 'mailparser'; @@ -6,270 +12,293 @@ import { logger } from '../../config/logger'; import { getThreadId } from './helpers/utils'; export class ImapConnector implements IEmailConnector { - private client: ImapFlow; - private newMaxUids: { [mailboxPath: string]: number; } = {}; - private isConnected = false; - private statusMessage: string | undefined; + private client: ImapFlow; + private newMaxUids: { [mailboxPath: string]: number } = {}; + private isConnected = false; + private statusMessage: string | undefined; - constructor(private credentials: GenericImapCredentials) { - this.client = this.createClient(); - } + constructor(private credentials: GenericImapCredentials) { + this.client = this.createClient(); + } - private createClient(): ImapFlow { - const client = new ImapFlow({ - host: this.credentials.host, - port: this.credentials.port, - secure: this.credentials.secure, - auth: { - user: this.credentials.username, - pass: this.credentials.password, - }, - logger: logger.child({ module: 'ImapFlow' }), - }); + private createClient(): ImapFlow { + const client = new ImapFlow({ + host: this.credentials.host, + port: this.credentials.port, + secure: this.credentials.secure, + auth: { + user: this.credentials.username, + pass: this.credentials.password, + }, + logger: logger.child({ module: 'ImapFlow' }), + }); - // Handles client-level errors, like unexpected disconnects, to prevent crashes. - client.on('error', (err) => { - logger.error({ err }, 'IMAP client error'); - this.isConnected = false; - }); + // Handles client-level errors, like unexpected disconnects, to prevent crashes. + client.on('error', (err) => { + logger.error({ err }, 'IMAP client error'); + this.isConnected = false; + }); - return client; - } + return client; + } - /** - * Establishes a connection to the IMAP server if not already connected. - */ - private async connect(): Promise { - if (this.isConnected && this.client.usable) { - return; - } + /** + * Establishes a connection to the IMAP server if not already connected. + */ + private async connect(): Promise { + if (this.isConnected && this.client.usable) { + return; + } - // If the client is not usable (e.g., after a logout), create a new one. - if (!this.client.usable) { - this.client = this.createClient(); - } + // If the client is not usable (e.g., after a logout), create a new one. + if (!this.client.usable) { + this.client = this.createClient(); + } - try { - await this.client.connect(); - this.isConnected = true; - } catch (err: any) { - this.isConnected = false; - logger.error({ err }, 'IMAP connection failed'); - if (err.responseText) { - throw new Error(`IMAP Connection Error: ${err.responseText}`); - } - throw err; - } - } + try { + await this.client.connect(); + this.isConnected = true; + } catch (err: any) { + this.isConnected = false; + logger.error({ err }, 'IMAP connection failed'); + if (err.responseText) { + throw new Error(`IMAP Connection Error: ${err.responseText}`); + } + throw err; + } + } - /** - * Disconnects from the IMAP server if the connection is active. - */ - private async disconnect(): Promise { - if (this.isConnected && this.client.usable) { - await this.client.logout(); - this.isConnected = false; - } - } + /** + * Disconnects from the IMAP server if the connection is active. + */ + private async disconnect(): Promise { + if (this.isConnected && this.client.usable) { + await this.client.logout(); + this.isConnected = false; + } + } - public async testConnection(): Promise { - try { - await this.connect(); - await this.disconnect(); - return true; - } catch (error) { - logger.error({ error }, 'Failed to verify IMAP connection'); - throw error; - } - } + public async testConnection(): Promise { + try { + await this.connect(); + await this.disconnect(); + return true; + } catch (error) { + logger.error({ error }, 'Failed to verify IMAP connection'); + throw error; + } + } - /** - * We understand that for IMAP inboxes, there is only one user, but we want the IMAP connector to be compatible with other connectors, we return the single user here. - * @returns An async generator that yields each user object. - */ - public async *listAllUsers(): AsyncGenerator { - try { - const emails: string[] = [this.returnImapUserEmail()]; - for (const [index, email] of emails.entries()) { - yield { - id: String(index), - primaryEmail: email, - displayName: email - }; + /** + * We understand that for IMAP inboxes, there is only one user, but we want the IMAP connector to be compatible with other connectors, we return the single user here. + * @returns An async generator that yields each user object. + */ + public async *listAllUsers(): AsyncGenerator { + try { + const emails: string[] = [this.returnImapUserEmail()]; + for (const [index, email] of emails.entries()) { + yield { + id: String(index), + primaryEmail: email, + displayName: email, + }; + } + } finally { + await this.disconnect(); + } + } - } - } finally { - await this.disconnect(); - } - } + public returnImapUserEmail(): string { + return this.credentials.username; + } - public returnImapUserEmail(): string { - return this.credentials.username; - } + /** + * Wraps an IMAP operation with a retry mechanism to handle transient network errors. + * @param action The async function to execute. + * @param maxRetries The maximum number of retries. + * @returns The result of the action. + */ + private async withRetry(action: () => Promise, maxRetries = 5): Promise { + for (let attempt = 1; attempt <= maxRetries; attempt++) { + try { + await this.connect(); + return await action(); + } catch (err: any) { + logger.error({ err, attempt }, `IMAP operation failed on attempt ${attempt}`); + this.isConnected = false; // Force reconnect on next attempt + if (attempt === maxRetries) { + logger.error({ err }, 'IMAP operation failed after all retries.'); + throw err; + } + // Wait for a short period before retrying + const delay = Math.pow(2, attempt) * 1000; + const jitter = Math.random() * 1000; + logger.info(`Retrying in ${Math.round((delay + jitter) / 1000)}s`); + await new Promise((resolve) => setTimeout(resolve, delay + jitter)); + } + } + // This line should be unreachable + throw new Error('IMAP operation failed after all retries.'); + } - /** - * Wraps an IMAP operation with a retry mechanism to handle transient network errors. - * @param action The async function to execute. - * @param maxRetries The maximum number of retries. - * @returns The result of the action. - */ - private async withRetry(action: () => Promise, maxRetries = 5): Promise { - for (let attempt = 1; attempt <= maxRetries; attempt++) { - try { - await this.connect(); - return await action(); - } catch (err: any) { - logger.error({ err, attempt }, `IMAP operation failed on attempt ${attempt}`); - this.isConnected = false; // Force reconnect on next attempt - if (attempt === maxRetries) { - logger.error({ err }, 'IMAP operation failed after all retries.'); - throw err; - } - // Wait for a short period before retrying - const delay = Math.pow(2, attempt) * 1000; - const jitter = Math.random() * 1000; - logger.info(`Retrying in ${Math.round((delay + jitter) / 1000)}s`); - await new Promise(resolve => setTimeout(resolve, delay + jitter)); - } - } - // This line should be unreachable - throw new Error('IMAP operation failed after all retries.'); - } + public async *fetchEmails( + userEmail: string, + syncState?: SyncState | null + ): AsyncGenerator { + // list all mailboxes first + const mailboxes = await this.withRetry(async () => await this.client.list()); + await this.disconnect(); - public async *fetchEmails(userEmail: string, syncState?: SyncState | null): AsyncGenerator { - // list all mailboxes first - const mailboxes = await this.withRetry(async () => await this.client.list()); - await this.disconnect(); + const processableMailboxes = mailboxes.filter((mailbox) => { + // filter out trash and all mail emails + if (mailbox.specialUse) { + const specialUse = mailbox.specialUse.toLowerCase(); + if (specialUse === '\\junk' || specialUse === '\\trash' || specialUse === '\\all') { + return false; + } + } + // Fallback to checking flags + if ( + mailbox.flags.has('\\Noselect') || + mailbox.flags.has('\\Trash') || + mailbox.flags.has('\\Junk') || + mailbox.flags.has('\\All') + ) { + return false; + } - const processableMailboxes = mailboxes.filter(mailbox => { - // filter out trash and all mail emails - if (mailbox.specialUse) { - const specialUse = mailbox.specialUse.toLowerCase(); - if (specialUse === '\\junk' || specialUse === '\\trash' || specialUse === '\\all') { - return false; - } - } - // Fallback to checking flags - if (mailbox.flags.has('\\Noselect') || mailbox.flags.has('\\Trash') || mailbox.flags.has('\\Junk') || mailbox.flags.has('\\All')) { - return false; - } + return true; + }); - return true; - }); + for (const mailboxInfo of processableMailboxes) { + const mailboxPath = mailboxInfo.path; + logger.info({ mailboxPath }, 'Processing mailbox'); - for (const mailboxInfo of processableMailboxes) { - const mailboxPath = mailboxInfo.path; - logger.info({ mailboxPath }, 'Processing mailbox'); + try { + const mailbox = await this.withRetry( + async () => await this.client.mailboxOpen(mailboxPath) + ); + const lastUid = syncState?.imap?.[mailboxPath]?.maxUid; + let currentMaxUid = lastUid || 0; - try { - const mailbox = await this.withRetry(async () => await this.client.mailboxOpen(mailboxPath)); - const lastUid = syncState?.imap?.[mailboxPath]?.maxUid; - let currentMaxUid = lastUid || 0; + if (mailbox.exists > 0) { + const lastMessage = await this.client.fetchOne(String(mailbox.exists), { + uid: true, + }); + if (lastMessage && lastMessage.uid > currentMaxUid) { + currentMaxUid = lastMessage.uid; + } + } + this.newMaxUids[mailboxPath] = currentMaxUid; - if (mailbox.exists > 0) { - const lastMessage = await this.client.fetchOne(String(mailbox.exists), { uid: true }); - if (lastMessage && lastMessage.uid > currentMaxUid) { - currentMaxUid = lastMessage.uid; - } - } - this.newMaxUids[mailboxPath] = currentMaxUid; + // Only fetch if the mailbox has messages, to avoid errors on empty mailboxes with some IMAP servers. + if (mailbox.exists > 0) { + const BATCH_SIZE = 250; // A configurable batch size + let startUid = (lastUid || 0) + 1; + const maxUidToFetch = currentMaxUid; - // Only fetch if the mailbox has messages, to avoid errors on empty mailboxes with some IMAP servers. - if (mailbox.exists > 0) { - const BATCH_SIZE = 250; // A configurable batch size - let startUid = (lastUid || 0) + 1; - const maxUidToFetch = currentMaxUid; + while (startUid <= maxUidToFetch) { + const endUid = Math.min(startUid + BATCH_SIZE - 1, maxUidToFetch); + const searchCriteria = { uid: `${startUid}:${endUid}` }; - while (startUid <= maxUidToFetch) { - const endUid = Math.min(startUid + BATCH_SIZE - 1, maxUidToFetch); - const searchCriteria = { uid: `${startUid}:${endUid}` }; + for await (const msg of this.client.fetch(searchCriteria, { + envelope: true, + source: true, + bodyStructure: true, + uid: true, + })) { + if (lastUid && msg.uid <= lastUid) { + continue; + } - for await (const msg of this.client.fetch(searchCriteria, { envelope: true, source: true, bodyStructure: true, uid: true })) { - if (lastUid && msg.uid <= lastUid) { - continue; - } + if (msg.uid > this.newMaxUids[mailboxPath]) { + this.newMaxUids[mailboxPath] = msg.uid; + } - if (msg.uid > this.newMaxUids[mailboxPath]) { - this.newMaxUids[mailboxPath] = msg.uid; - } + logger.debug({ mailboxPath, uid: msg.uid }, 'Processing message'); - logger.debug({ mailboxPath, uid: msg.uid }, 'Processing message'); + if (msg.envelope && msg.source) { + try { + yield await this.parseMessage(msg, mailboxPath); + } catch (err: any) { + logger.error( + { err, mailboxPath, uid: msg.uid }, + 'Failed to parse message' + ); + throw err; + } + } + } - if (msg.envelope && msg.source) { - try { - yield await this.parseMessage(msg, mailboxPath); - } catch (err: any) { - logger.error({ err, mailboxPath, uid: msg.uid }, 'Failed to parse message'); - throw err; - } - } - } + // Move to the next batch + startUid = endUid + 1; + } + } + } catch (err: any) { + logger.error({ err, mailboxPath }, 'Failed to process mailbox'); + // Check if the error indicates a persistent failure after retries + if (err.message.includes('IMAP operation failed after all retries')) { + this.statusMessage = + 'Sync paused due to reaching the mail server rate limit. The process will automatically resume later.'; + } + } finally { + await this.disconnect(); + } + } + } - // Move to the next batch - startUid = endUid + 1; - } - } - } catch (err: any) { - logger.error({ err, mailboxPath }, 'Failed to process mailbox'); - // Check if the error indicates a persistent failure after retries - if (err.message.includes('IMAP operation failed after all retries')) { - this.statusMessage = 'Sync paused due to reaching the mail server rate limit. The process will automatically resume later.'; - } - } - finally { - await this.disconnect(); - } - } - } + private async parseMessage(msg: any, mailboxPath: string): Promise { + const parsedEmail: ParsedMail = await simpleParser(msg.source); + const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ + filename: attachment.filename || 'untitled', + contentType: attachment.contentType, + size: attachment.size, + content: attachment.content as Buffer, + })); - private async parseMessage(msg: any, mailboxPath: string): Promise { - const parsedEmail: ParsedMail = await simpleParser(msg.source); - const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ - filename: attachment.filename || 'untitled', - contentType: attachment.contentType, - size: attachment.size, - content: attachment.content as Buffer - })); + const mapAddresses = ( + addresses: AddressObject | AddressObject[] | undefined + ): EmailAddress[] => { + if (!addresses) return []; + const addressArray = Array.isArray(addresses) ? addresses : [addresses]; + return addressArray.flatMap((a) => + a.value.map((v) => ({ name: v.name, address: v.address || '' })) + ); + }; - const mapAddresses = (addresses: AddressObject | AddressObject[] | undefined): EmailAddress[] => { - if (!addresses) return []; - const addressArray = Array.isArray(addresses) ? addresses : [addresses]; - return addressArray.flatMap(a => a.value.map(v => ({ name: v.name, address: v.address || '' }))); - }; + const threadId = getThreadId(parsedEmail.headers); - const threadId = getThreadId(parsedEmail.headers); + return { + id: parsedEmail.messageId || msg.uid.toString(), + threadId: threadId, + from: mapAddresses(parsedEmail.from), + to: mapAddresses(parsedEmail.to), + cc: mapAddresses(parsedEmail.cc), + bcc: mapAddresses(parsedEmail.bcc), + subject: parsedEmail.subject || '', + body: parsedEmail.text || '', + html: parsedEmail.html || '', + headers: parsedEmail.headers, + attachments, + receivedAt: parsedEmail.date || new Date(), + eml: msg.source, + path: mailboxPath, + }; + } - return { - id: parsedEmail.messageId || msg.uid.toString(), - threadId: threadId, - from: mapAddresses(parsedEmail.from), - to: mapAddresses(parsedEmail.to), - cc: mapAddresses(parsedEmail.cc), - bcc: mapAddresses(parsedEmail.bcc), - subject: parsedEmail.subject || '', - body: parsedEmail.text || '', - html: parsedEmail.html || '', - headers: parsedEmail.headers, - attachments, - receivedAt: parsedEmail.date || new Date(), - eml: msg.source, - path: mailboxPath - }; - } + public getUpdatedSyncState(): SyncState { + const imapSyncState: { [mailboxPath: string]: { maxUid: number } } = {}; + for (const [path, uid] of Object.entries(this.newMaxUids)) { + imapSyncState[path] = { maxUid: uid }; + } + const syncState: SyncState = { + imap: imapSyncState, + }; - public getUpdatedSyncState(): SyncState { - const imapSyncState: { [mailboxPath: string]: { maxUid: number; }; } = {}; - for (const [path, uid] of Object.entries(this.newMaxUids)) { - imapSyncState[path] = { maxUid: uid }; - } - const syncState: SyncState = { - imap: imapSyncState - }; + if (this.statusMessage) { + syncState.statusMessage = this.statusMessage; + } - if (this.statusMessage) { - syncState.statusMessage = this.statusMessage; - } - - return syncState; - } -} \ No newline at end of file + return syncState; + } +} diff --git a/packages/backend/src/services/ingestion-connectors/MicrosoftConnector.ts b/packages/backend/src/services/ingestion-connectors/MicrosoftConnector.ts index a03eaa0..bf290a0 100644 --- a/packages/backend/src/services/ingestion-connectors/MicrosoftConnector.ts +++ b/packages/backend/src/services/ingestion-connectors/MicrosoftConnector.ts @@ -1,10 +1,10 @@ import 'cross-fetch/polyfill'; import type { - Microsoft365Credentials, - EmailObject, - EmailAddress, - SyncState, - MailboxUser + Microsoft365Credentials, + EmailObject, + EmailAddress, + SyncState, + MailboxUser, } from '@open-archiver/types'; import type { IEmailConnector } from '../EmailProviderFactory'; import { logger } from '../../config/logger'; @@ -19,285 +19,319 @@ import type { AuthProvider } from '@microsoft/microsoft-graph-client'; * to access data on behalf of the organization. */ export class MicrosoftConnector implements IEmailConnector { - private credentials: Microsoft365Credentials; - private graphClient: Client; - // Store delta tokens for each folder during a sync operation. - private newDeltaTokens: { [folderId: string]: string; }; + private credentials: Microsoft365Credentials; + private graphClient: Client; + // Store delta tokens for each folder during a sync operation. + private newDeltaTokens: { [folderId: string]: string }; - constructor(credentials: Microsoft365Credentials) { - this.credentials = credentials; - this.newDeltaTokens = {}; // Initialize as an empty object + constructor(credentials: Microsoft365Credentials) { + this.credentials = credentials; + this.newDeltaTokens = {}; // Initialize as an empty object - const msalConfig: Configuration = { - auth: { - clientId: this.credentials.clientId, - authority: `https://login.microsoftonline.com/${this.credentials.tenantId}`, - clientSecret: this.credentials.clientSecret, - }, - system: { - loggerOptions: { - loggerCallback(loglevel, message, containsPii) { - if (containsPii) return; - switch (loglevel) { - case LogLevel.Error: - logger.error(message); - return; - case LogLevel.Warning: - logger.warn(message); - return; - case LogLevel.Info: - logger.info(message); - return; - case LogLevel.Verbose: - logger.debug(message); - return; - } - }, - piiLoggingEnabled: false, - logLevel: LogLevel.Warning, - } - } - }; + const msalConfig: Configuration = { + auth: { + clientId: this.credentials.clientId, + authority: `https://login.microsoftonline.com/${this.credentials.tenantId}`, + clientSecret: this.credentials.clientSecret, + }, + system: { + loggerOptions: { + loggerCallback(loglevel, message, containsPii) { + if (containsPii) return; + switch (loglevel) { + case LogLevel.Error: + logger.error(message); + return; + case LogLevel.Warning: + logger.warn(message); + return; + case LogLevel.Info: + logger.info(message); + return; + case LogLevel.Verbose: + logger.debug(message); + return; + } + }, + piiLoggingEnabled: false, + logLevel: LogLevel.Warning, + }, + }, + }; - const msalClient = new ConfidentialClientApplication(msalConfig); + const msalClient = new ConfidentialClientApplication(msalConfig); - const authProvider: AuthProvider = async (done) => { - try { - const response = await msalClient.acquireTokenByClientCredential({ - scopes: ['https://graph.microsoft.com/.default'], - }); - if (!response?.accessToken) { - throw new Error('Failed to acquire access token.'); - } - done(null, response.accessToken); - } catch (error) { - logger.error({ err: error }, 'Failed to acquire Microsoft Graph access token'); - done(error, null); - } - }; + const authProvider: AuthProvider = async (done) => { + try { + const response = await msalClient.acquireTokenByClientCredential({ + scopes: ['https://graph.microsoft.com/.default'], + }); + if (!response?.accessToken) { + throw new Error('Failed to acquire access token.'); + } + done(null, response.accessToken); + } catch (error) { + logger.error({ err: error }, 'Failed to acquire Microsoft Graph access token'); + done(error, null); + } + }; - this.graphClient = Client.init({ authProvider }); - } + this.graphClient = Client.init({ authProvider }); + } - /** - * Tests the connection and authentication by attempting to list the first user - * from the directory. - */ - public async testConnection(): Promise { - try { - await this.graphClient.api('/users').top(1).get(); - logger.info('Microsoft 365 connection test successful.'); - return true; - } catch (error) { - logger.error({ err: error }, 'Failed to verify Microsoft 365 connection'); - throw error; - } - } + /** + * Tests the connection and authentication by attempting to list the first user + * from the directory. + */ + public async testConnection(): Promise { + try { + await this.graphClient.api('/users').top(1).get(); + logger.info('Microsoft 365 connection test successful.'); + return true; + } catch (error) { + logger.error({ err: error }, 'Failed to verify Microsoft 365 connection'); + throw error; + } + } - /** - * Lists all users in the Microsoft 365 tenant. - * This method handles pagination to retrieve the complete list of users. - * @returns An async generator that yields each user object. - */ - public async *listAllUsers(): AsyncGenerator { - let request = this.graphClient.api('/users').select('id,userPrincipalName,displayName'); + /** + * Lists all users in the Microsoft 365 tenant. + * This method handles pagination to retrieve the complete list of users. + * @returns An async generator that yields each user object. + */ + public async *listAllUsers(): AsyncGenerator { + let request = this.graphClient.api('/users').select('id,userPrincipalName,displayName'); - try { - let response = await request.get(); - while (response) { - for (const user of response.value as User[]) { - if (user.id && user.userPrincipalName && user.displayName) { - yield { - id: user.id, - primaryEmail: user.userPrincipalName, - displayName: user.displayName - }; - } - } + try { + let response = await request.get(); + while (response) { + for (const user of response.value as User[]) { + if (user.id && user.userPrincipalName && user.displayName) { + yield { + id: user.id, + primaryEmail: user.userPrincipalName, + displayName: user.displayName, + }; + } + } - if (response['@odata.nextLink']) { - response = await this.graphClient.api(response['@odata.nextLink']).get(); - } else { - break; - } - } - } catch (error) { - logger.error({ err: error }, 'Failed to list all users from Microsoft 365'); - throw error; - } - } + if (response['@odata.nextLink']) { + response = await this.graphClient.api(response['@odata.nextLink']).get(); + } else { + break; + } + } + } catch (error) { + logger.error({ err: error }, 'Failed to list all users from Microsoft 365'); + throw error; + } + } - /** - * Fetches emails for a single user by iterating through all mail folders and - * performing a delta query on each. - * @param userEmail The user principal name or ID of the user. - * @param syncState Optional state containing the deltaTokens for each folder. - * @returns An async generator that yields each raw email object. - */ - public async *fetchEmails( - userEmail: string, - syncState?: SyncState | null - ): AsyncGenerator { - this.newDeltaTokens = syncState?.microsoft?.[userEmail]?.deltaTokens || {}; + /** + * Fetches emails for a single user by iterating through all mail folders and + * performing a delta query on each. + * @param userEmail The user principal name or ID of the user. + * @param syncState Optional state containing the deltaTokens for each folder. + * @returns An async generator that yields each raw email object. + */ + public async *fetchEmails( + userEmail: string, + syncState?: SyncState | null + ): AsyncGenerator { + this.newDeltaTokens = syncState?.microsoft?.[userEmail]?.deltaTokens || {}; - try { - const folders = this.listAllFolders(userEmail); - for await (const folder of folders) { - if (folder.id && folder.path) { - logger.info({ userEmail, folderId: folder.id, folderName: folder.displayName }, 'Syncing folder'); - yield* this.syncFolder(userEmail, folder.id, folder.path, this.newDeltaTokens[folder.id]); - } - } - } catch (error) { - logger.error({ err: error, userEmail }, 'Failed to fetch emails from Microsoft 365'); - throw error; - } - } + try { + const folders = this.listAllFolders(userEmail); + for await (const folder of folders) { + if (folder.id && folder.path) { + logger.info( + { userEmail, folderId: folder.id, folderName: folder.displayName }, + 'Syncing folder' + ); + yield* this.syncFolder( + userEmail, + folder.id, + folder.path, + this.newDeltaTokens[folder.id] + ); + } + } + } catch (error) { + logger.error({ err: error, userEmail }, 'Failed to fetch emails from Microsoft 365'); + throw error; + } + } - /** - * Lists all mail folders for a given user. - * @param userEmail The user principal name or ID. - * @returns An async generator that yields each mail folder. - */ - private async *listAllFolders(userEmail: string, parentFolderId?: string, currentPath = ''): AsyncGenerator { - const requestUrl = parentFolderId - ? `/users/${userEmail}/mailFolders/${parentFolderId}/childFolders` - : `/users/${userEmail}/mailFolders`; + /** + * Lists all mail folders for a given user. + * @param userEmail The user principal name or ID. + * @returns An async generator that yields each mail folder. + */ + private async *listAllFolders( + userEmail: string, + parentFolderId?: string, + currentPath = '' + ): AsyncGenerator { + const requestUrl = parentFolderId + ? `/users/${userEmail}/mailFolders/${parentFolderId}/childFolders` + : `/users/${userEmail}/mailFolders`; - try { - let response = await this.graphClient.api(requestUrl).get(); + try { + let response = await this.graphClient.api(requestUrl).get(); - while (response) { - for (const folder of response.value as MailFolder[]) { - const newPath = currentPath ? `${currentPath}/${folder.displayName || ''}` : folder.displayName || ''; - yield { ...folder, path: newPath || '' }; + while (response) { + for (const folder of response.value as MailFolder[]) { + const newPath = currentPath + ? `${currentPath}/${folder.displayName || ''}` + : folder.displayName || ''; + yield { ...folder, path: newPath || '' }; - if (folder.childFolderCount && folder.childFolderCount > 0) { - yield* this.listAllFolders(userEmail, folder.id, newPath); - } - } + if (folder.childFolderCount && folder.childFolderCount > 0) { + yield* this.listAllFolders(userEmail, folder.id, newPath); + } + } - if (response['@odata.nextLink']) { - response = await this.graphClient.api(response['@odata.nextLink']).get(); - } else { - break; - } - } - } catch (error) { - logger.error({ err: error, userEmail }, 'Failed to list mail folders'); - throw error; - } - } + if (response['@odata.nextLink']) { + response = await this.graphClient.api(response['@odata.nextLink']).get(); + } else { + break; + } + } + } catch (error) { + logger.error({ err: error, userEmail }, 'Failed to list mail folders'); + throw error; + } + } - /** - * Performs a delta sync on a single mail folder. - * @param userEmail The user's email. - * @param folderId The ID of the folder to sync. - * @param deltaToken The existing delta token for this folder, if any. - * @returns An async generator that yields email objects. - */ - private async *syncFolder( - userEmail: string, - folderId: string, - path: string, - deltaToken?: string - ): AsyncGenerator { - let requestUrl: string | undefined; + /** + * Performs a delta sync on a single mail folder. + * @param userEmail The user's email. + * @param folderId The ID of the folder to sync. + * @param deltaToken The existing delta token for this folder, if any. + * @returns An async generator that yields email objects. + */ + private async *syncFolder( + userEmail: string, + folderId: string, + path: string, + deltaToken?: string + ): AsyncGenerator { + let requestUrl: string | undefined; - if (deltaToken) { - // Continuous sync - requestUrl = deltaToken; - } else { - // Initial sync - requestUrl = `/users/${userEmail}/mailFolders/${folderId}/messages/delta`; - } + if (deltaToken) { + // Continuous sync + requestUrl = deltaToken; + } else { + // Initial sync + requestUrl = `/users/${userEmail}/mailFolders/${folderId}/messages/delta`; + } - while (requestUrl) { - try { - const response = await this.graphClient.api(requestUrl) - .select('id,conversationId,@removed') - .get(); + while (requestUrl) { + try { + const response = await this.graphClient + .api(requestUrl) + .select('id,conversationId,@removed') + .get(); - for (const message of response.value) { - if (message.id && !(message)['@removed']) { - const rawEmail = await this.getRawEmail(userEmail, message.id); - if (rawEmail) { - const emailObject = await this.parseEmail(rawEmail, message.id, userEmail, path); - emailObject.threadId = message.conversationId; // Add conversationId as threadId - yield emailObject; - } - } - } + for (const message of response.value) { + if (message.id && !message['@removed']) { + const rawEmail = await this.getRawEmail(userEmail, message.id); + if (rawEmail) { + const emailObject = await this.parseEmail( + rawEmail, + message.id, + userEmail, + path + ); + emailObject.threadId = message.conversationId; // Add conversationId as threadId + yield emailObject; + } + } + } - if (response['@odata.deltaLink']) { - this.newDeltaTokens[folderId] = response['@odata.deltaLink']; - } + if (response['@odata.deltaLink']) { + this.newDeltaTokens[folderId] = response['@odata.deltaLink']; + } - requestUrl = response['@odata.nextLink']; - } catch (error) { - logger.error({ err: error, userEmail, folderId }, 'Failed to sync mail folder'); - // Continue to the next folder if one fails - return; - } - } - } + requestUrl = response['@odata.nextLink']; + } catch (error) { + logger.error({ err: error, userEmail, folderId }, 'Failed to sync mail folder'); + // Continue to the next folder if one fails + return; + } + } + } - private async getRawEmail(userEmail: string, messageId: string): Promise { - try { - const response = await this.graphClient.api(`/users/${userEmail}/messages/${messageId}/$value`).getStream(); - const chunks: any[] = []; - for await (const chunk of response) { - chunks.push(chunk); - } - return Buffer.concat(chunks); - } catch (error) { - logger.error({ err: error, userEmail, messageId }, 'Failed to fetch raw email content.'); - return null; - } - } + private async getRawEmail(userEmail: string, messageId: string): Promise { + try { + const response = await this.graphClient + .api(`/users/${userEmail}/messages/${messageId}/$value`) + .getStream(); + const chunks: any[] = []; + for await (const chunk of response) { + chunks.push(chunk); + } + return Buffer.concat(chunks); + } catch (error) { + logger.error( + { err: error, userEmail, messageId }, + 'Failed to fetch raw email content.' + ); + return null; + } + } - private async parseEmail(rawEmail: Buffer, messageId: string, userEmail: string, path: string): Promise { - const parsedEmail: ParsedMail = await simpleParser(rawEmail); - const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ - filename: attachment.filename || 'untitled', - contentType: attachment.contentType, - size: attachment.size, - content: attachment.content as Buffer - })); - const mapAddresses = (addresses: AddressObject | AddressObject[] | undefined): EmailAddress[] => { - if (!addresses) return []; - const addressArray = Array.isArray(addresses) ? addresses : [addresses]; - return addressArray.flatMap(a => a.value.map(v => ({ name: v.name, address: v.address || '' }))); - }; + private async parseEmail( + rawEmail: Buffer, + messageId: string, + userEmail: string, + path: string + ): Promise { + const parsedEmail: ParsedMail = await simpleParser(rawEmail); + const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ + filename: attachment.filename || 'untitled', + contentType: attachment.contentType, + size: attachment.size, + content: attachment.content as Buffer, + })); + const mapAddresses = ( + addresses: AddressObject | AddressObject[] | undefined + ): EmailAddress[] => { + if (!addresses) return []; + const addressArray = Array.isArray(addresses) ? addresses : [addresses]; + return addressArray.flatMap((a) => + a.value.map((v) => ({ name: v.name, address: v.address || '' })) + ); + }; - return { - id: messageId, - userEmail: userEmail, - eml: rawEmail, - from: mapAddresses(parsedEmail.from), - to: mapAddresses(parsedEmail.to), - cc: mapAddresses(parsedEmail.cc), - bcc: mapAddresses(parsedEmail.bcc), - subject: parsedEmail.subject || '', - body: parsedEmail.text || '', - html: parsedEmail.html || '', - headers: parsedEmail.headers, - attachments, - receivedAt: parsedEmail.date || new Date(), - path - }; - } + return { + id: messageId, + userEmail: userEmail, + eml: rawEmail, + from: mapAddresses(parsedEmail.from), + to: mapAddresses(parsedEmail.to), + cc: mapAddresses(parsedEmail.cc), + bcc: mapAddresses(parsedEmail.bcc), + subject: parsedEmail.subject || '', + body: parsedEmail.text || '', + html: parsedEmail.html || '', + headers: parsedEmail.headers, + attachments, + receivedAt: parsedEmail.date || new Date(), + path, + }; + } - public getUpdatedSyncState(userEmail: string): SyncState { - if (Object.keys(this.newDeltaTokens).length === 0) { - return {}; - } - return { - microsoft: { - [userEmail]: { - deltaTokens: this.newDeltaTokens - } - } - }; - } + public getUpdatedSyncState(userEmail: string): SyncState { + if (Object.keys(this.newDeltaTokens).length === 0) { + return {}; + } + return { + microsoft: { + [userEmail]: { + deltaTokens: this.newDeltaTokens, + }, + }, + }; + } } diff --git a/packages/backend/src/services/ingestion-connectors/PSTConnector.ts b/packages/backend/src/services/ingestion-connectors/PSTConnector.ts index 2989624..f3a0376 100644 --- a/packages/backend/src/services/ingestion-connectors/PSTConnector.ts +++ b/packages/backend/src/services/ingestion-connectors/PSTConnector.ts @@ -1,4 +1,10 @@ -import type { PSTImportCredentials, EmailObject, EmailAddress, SyncState, MailboxUser } from '@open-archiver/types'; +import type { + PSTImportCredentials, + EmailObject, + EmailAddress, + SyncState, + MailboxUser, +} from '@open-archiver/types'; import type { IEmailConnector } from '../EmailProviderFactory'; import { PSTFile, PSTFolder, PSTMessage } from 'pst-extractor'; import { simpleParser, ParsedMail, Attachment, AddressObject } from 'mailparser'; @@ -9,334 +15,374 @@ import { Readable } from 'stream'; import { createHash } from 'crypto'; const streamToBuffer = (stream: Readable): Promise => { - return new Promise((resolve, reject) => { - const chunks: Buffer[] = []; - stream.on('data', (chunk) => chunks.push(chunk)); - stream.on('error', reject); - stream.on('end', () => resolve(Buffer.concat(chunks))); - }); + return new Promise((resolve, reject) => { + const chunks: Buffer[] = []; + stream.on('data', (chunk) => chunks.push(chunk)); + stream.on('error', reject); + stream.on('end', () => resolve(Buffer.concat(chunks))); + }); }; // We have to hardcode names for deleted and trash folders here as current lib doesn't support looking into PST properties. const DELETED_FOLDERS = new Set([ - // English - 'deleted items', 'trash', - // Spanish - 'elementos eliminados', 'papelera', - // French - 'éléments supprimés', 'corbeille', - // German - 'gelöschte elemente', 'papierkorb', - // Italian - 'posta eliminata', 'cestino', - // Portuguese - 'itens excluídos', 'lixo', - // Dutch - 'verwijderde items', 'prullenbak', - // Russian - 'удаленные', 'корзина', - // Polish - 'usunięte elementy', 'kosz', - // Japanese - '削除済みアイテム', - // Czech - 'odstraněná pošta', 'koš', - // Estonian - 'kustutatud kirjad', 'prügikast', - // Swedish - 'borttagna objekt', 'skräp', - // Danish - 'slettet post', 'papirkurv', - // Norwegian - 'slettede elementer', - // Finnish - 'poistetut', 'roskakori' + // English + 'deleted items', + 'trash', + // Spanish + 'elementos eliminados', + 'papelera', + // French + 'éléments supprimés', + 'corbeille', + // German + 'gelöschte elemente', + 'papierkorb', + // Italian + 'posta eliminata', + 'cestino', + // Portuguese + 'itens excluídos', + 'lixo', + // Dutch + 'verwijderde items', + 'prullenbak', + // Russian + 'удаленные', + 'корзина', + // Polish + 'usunięte elementy', + 'kosz', + // Japanese + '削除済みアイテム', + // Czech + 'odstraněná pošta', + 'koš', + // Estonian + 'kustutatud kirjad', + 'prügikast', + // Swedish + 'borttagna objekt', + 'skräp', + // Danish + 'slettet post', + 'papirkurv', + // Norwegian + 'slettede elementer', + // Finnish + 'poistetut', + 'roskakori', ]); const JUNK_FOLDERS = new Set([ - // English - 'junk email', 'spam', - // Spanish - 'correo no deseado', - // French - 'courrier indésirable', - // German - 'junk-e-mail', - // Italian - 'posta indesiderata', - // Portuguese - 'lixo eletrônico', - // Dutch - 'ongewenste e-mail', - // Russian - 'нежелательная почта', 'спам', - // Polish - 'wiadomości-śmieci', - // Japanese - '迷惑メール', 'スパム', - // Czech - 'nevyžádaná pošta', - // Estonian - 'rämpspost', - // Swedish - 'skräppost', - // Danish - 'uønsket post', - // Norwegian - 'søppelpost', - // Finnish - 'roskaposti' + // English + 'junk email', + 'spam', + // Spanish + 'correo no deseado', + // French + 'courrier indésirable', + // German + 'junk-e-mail', + // Italian + 'posta indesiderata', + // Portuguese + 'lixo eletrônico', + // Dutch + 'ongewenste e-mail', + // Russian + 'нежелательная почта', + 'спам', + // Polish + 'wiadomości-śmieci', + // Japanese + '迷惑メール', + 'スパム', + // Czech + 'nevyžádaná pošta', + // Estonian + 'rämpspost', + // Swedish + 'skräppost', + // Danish + 'uønsket post', + // Norwegian + 'søppelpost', + // Finnish + 'roskaposti', ]); export class PSTConnector implements IEmailConnector { - private storage: StorageService; - private pstFile: PSTFile | null = null; + private storage: StorageService; + private pstFile: PSTFile | null = null; - constructor(private credentials: PSTImportCredentials) { - this.storage = new StorageService(); - } + constructor(private credentials: PSTImportCredentials) { + this.storage = new StorageService(); + } - private async loadPstFile(): Promise { - if (this.pstFile) { - return this.pstFile; - } - const fileStream = await this.storage.get(this.credentials.uploadedFilePath); - const buffer = await streamToBuffer(fileStream as Readable); - this.pstFile = new PSTFile(buffer); - return this.pstFile; - } + private async loadPstFile(): Promise { + if (this.pstFile) { + return this.pstFile; + } + const fileStream = await this.storage.get(this.credentials.uploadedFilePath); + const buffer = await streamToBuffer(fileStream as Readable); + this.pstFile = new PSTFile(buffer); + return this.pstFile; + } - public async testConnection(): Promise { - try { - if (!this.credentials.uploadedFilePath) { - throw Error("PST file path not provided."); - } - if (!this.credentials.uploadedFilePath.includes('.pst')) { - throw Error("Provided file is not in the PST format."); - } - const fileExist = await this.storage.exists(this.credentials.uploadedFilePath); - if (!fileExist) { - throw Error("PST file upload not finished yet, please wait."); - } + public async testConnection(): Promise { + try { + if (!this.credentials.uploadedFilePath) { + throw Error('PST file path not provided.'); + } + if (!this.credentials.uploadedFilePath.includes('.pst')) { + throw Error('Provided file is not in the PST format.'); + } + const fileExist = await this.storage.exists(this.credentials.uploadedFilePath); + if (!fileExist) { + throw Error('PST file upload not finished yet, please wait.'); + } - return true; - } catch (error) { - logger.error({ error, credentials: this.credentials }, 'PST file validation failed.'); - throw error; - } - } + return true; + } catch (error) { + logger.error({ error, credentials: this.credentials }, 'PST file validation failed.'); + throw error; + } + } - /** - * Lists mailboxes within the PST. It treats each top-level folder - * as a distinct mailbox, allowing it to handle PSTs that have been - * consolidated from multiple sources. - */ - public async *listAllUsers(): AsyncGenerator { - let pstFile: PSTFile | null = null; - try { - pstFile = await this.loadPstFile(); - const root = pstFile.getRootFolder(); - const displayName: string = root.displayName || pstFile.pstFilename || String(new Date().getTime()); - logger.info(`Found potential mailbox: ${displayName}`); - const constructedPrimaryEmail = `${displayName.replace(/ /g, '.').toLowerCase()}@pst.local`; - yield { - id: constructedPrimaryEmail, - // We will address the primaryEmail problem in the next section. - primaryEmail: constructedPrimaryEmail, - displayName: displayName, - }; - } catch (error) { - logger.error({ error }, 'Failed to list users from PST file.'); - pstFile?.close(); - throw error; - } finally { - pstFile?.close(); - } - } + /** + * Lists mailboxes within the PST. It treats each top-level folder + * as a distinct mailbox, allowing it to handle PSTs that have been + * consolidated from multiple sources. + */ + public async *listAllUsers(): AsyncGenerator { + let pstFile: PSTFile | null = null; + try { + pstFile = await this.loadPstFile(); + const root = pstFile.getRootFolder(); + const displayName: string = + root.displayName || pstFile.pstFilename || String(new Date().getTime()); + logger.info(`Found potential mailbox: ${displayName}`); + const constructedPrimaryEmail = `${displayName.replace(/ /g, '.').toLowerCase()}@pst.local`; + yield { + id: constructedPrimaryEmail, + // We will address the primaryEmail problem in the next section. + primaryEmail: constructedPrimaryEmail, + displayName: displayName, + }; + } catch (error) { + logger.error({ error }, 'Failed to list users from PST file.'); + pstFile?.close(); + throw error; + } finally { + pstFile?.close(); + } + } - public async *fetchEmails(userEmail: string, syncState?: SyncState | null): AsyncGenerator { - let pstFile: PSTFile | null = null; - try { - pstFile = await this.loadPstFile(); - const root = pstFile.getRootFolder(); - yield* this.processFolder(root, '', userEmail); - } catch (error) { - logger.error({ error }, 'Failed to fetch email.'); - pstFile?.close(); - throw error; - } - finally { + public async *fetchEmails( + userEmail: string, + syncState?: SyncState | null + ): AsyncGenerator { + let pstFile: PSTFile | null = null; + try { + pstFile = await this.loadPstFile(); + const root = pstFile.getRootFolder(); + yield* this.processFolder(root, '', userEmail); + } catch (error) { + logger.error({ error }, 'Failed to fetch email.'); + pstFile?.close(); + throw error; + } finally { + pstFile?.close(); + } + } - pstFile?.close(); - } - } + private async *processFolder( + folder: PSTFolder, + currentPath: string, + userEmail: string + ): AsyncGenerator { + const folderName = folder.displayName.toLowerCase(); + if (DELETED_FOLDERS.has(folderName) || JUNK_FOLDERS.has(folderName)) { + logger.info(`Skipping folder: ${folder.displayName}`); + return; + } - private async *processFolder(folder: PSTFolder, currentPath: string, userEmail: string): AsyncGenerator { - const folderName = folder.displayName.toLowerCase(); - if (DELETED_FOLDERS.has(folderName) || JUNK_FOLDERS.has(folderName)) { - logger.info(`Skipping folder: ${folder.displayName}`); - return; - } + const newPath = currentPath ? `${currentPath}/${folder.displayName}` : folder.displayName; - const newPath = currentPath ? `${currentPath}/${folder.displayName}` : folder.displayName; + if (folder.contentCount > 0) { + let email: PSTMessage | null = folder.getNextChild(); + while (email != null) { + yield await this.parseMessage(email, newPath, userEmail); + try { + email = folder.getNextChild(); + } catch (error) { + console.warn("Folder doesn't have child"); + email = null; + } + } + } - if (folder.contentCount > 0) { - let email: PSTMessage | null = folder.getNextChild(); - while (email != null) { - yield await this.parseMessage(email, newPath, userEmail); - try { - email = folder.getNextChild(); - } catch (error) { - console.warn("Folder doesn't have child"); - email = null; - } - } - } + if (folder.hasSubfolders) { + for (const subFolder of folder.getSubFolders()) { + yield* this.processFolder(subFolder, newPath, userEmail); + } + } + } - if (folder.hasSubfolders) { - for (const subFolder of folder.getSubFolders()) { - yield* this.processFolder(subFolder, newPath, userEmail); - } - } - } + private async parseMessage( + msg: PSTMessage, + path: string, + userEmail: string + ): Promise { + const emlContent = await this.constructEml(msg); + const emlBuffer = Buffer.from(emlContent, 'utf-8'); + const parsedEmail: ParsedMail = await simpleParser(emlBuffer); - private async parseMessage(msg: PSTMessage, path: string, userEmail: string): Promise { - const emlContent = await this.constructEml(msg); - const emlBuffer = Buffer.from(emlContent, 'utf-8'); - const parsedEmail: ParsedMail = await simpleParser(emlBuffer); + const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ + filename: attachment.filename || 'untitled', + contentType: attachment.contentType, + size: attachment.size, + content: attachment.content as Buffer, + })); - const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({ - filename: attachment.filename || 'untitled', - contentType: attachment.contentType, - size: attachment.size, - content: attachment.content as Buffer - })); + const mapAddresses = ( + addresses: AddressObject | AddressObject[] | undefined + ): EmailAddress[] => { + if (!addresses) return []; + const addressArray = Array.isArray(addresses) ? addresses : [addresses]; + return addressArray.flatMap((a) => + a.value.map((v) => ({ + name: v.name, + address: v.address?.replaceAll(`'`, '') || '', + })) + ); + }; - const mapAddresses = (addresses: AddressObject | AddressObject[] | undefined): EmailAddress[] => { - if (!addresses) return []; - const addressArray = Array.isArray(addresses) ? addresses : [addresses]; - return addressArray.flatMap(a => a.value.map(v => ({ name: v.name, address: v.address?.replaceAll(`'`, '') || '' }))); - }; + const from = mapAddresses(parsedEmail.from); + if (from.length === 0) { + from.push({ name: 'No Sender', address: 'No Sender' }); + } - const from = mapAddresses(parsedEmail.from); - if (from.length === 0) { - from.push({ name: 'No Sender', address: 'No Sender' }); - } + const threadId = getThreadId(parsedEmail.headers); + let messageId = msg.internetMessageId; + // generate a unique ID for this message - const threadId = getThreadId(parsedEmail.headers); - let messageId = msg.internetMessageId; - // generate a unique ID for this message + if (!messageId) { + messageId = `generated-${createHash('sha256') + .update( + emlBuffer ?? Buffer.from(parsedEmail.text || parsedEmail.html || '', 'utf-8') + ) + .digest('hex')}-${createHash('sha256') + .update(emlBuffer ?? Buffer.from(msg.subject || '', 'utf-8')) + .digest('hex')}-${msg.clientSubmitTime?.getTime()}`; + } + return { + id: messageId, + threadId: threadId, + from, + to: mapAddresses(parsedEmail.to), + cc: mapAddresses(parsedEmail.cc), + bcc: mapAddresses(parsedEmail.bcc), + subject: parsedEmail.subject || '', + body: parsedEmail.text || '', + html: parsedEmail.html || '', + headers: parsedEmail.headers, + attachments, + receivedAt: parsedEmail.date || new Date(), + eml: emlBuffer, + path, + }; + } - if (!messageId) { - messageId = `generated-${createHash('sha256').update(emlBuffer ?? Buffer.from(parsedEmail.text || parsedEmail.html || '', 'utf-8')).digest('hex')}-${createHash('sha256').update(emlBuffer ?? Buffer.from(msg.subject || '', 'utf-8')).digest('hex')}-${msg.clientSubmitTime?.getTime()}`; - } - return { - id: messageId, - threadId: threadId, - from, - to: mapAddresses(parsedEmail.to), - cc: mapAddresses(parsedEmail.cc), - bcc: mapAddresses(parsedEmail.bcc), - subject: parsedEmail.subject || '', - body: parsedEmail.text || '', - html: parsedEmail.html || '', - headers: parsedEmail.headers, - attachments, - receivedAt: parsedEmail.date || new Date(), - eml: emlBuffer, - path - }; - } + private async constructEml(msg: PSTMessage): Promise { + let eml = ''; + const boundary = '----boundary-openarchiver'; + const altBoundary = '----boundary-openarchiver_alt'; - private async constructEml(msg: PSTMessage): Promise { - let eml = ''; - const boundary = '----boundary-openarchiver'; - const altBoundary = '----boundary-openarchiver_alt'; + let headers = ''; - let headers = ''; + if (msg.senderName || msg.senderEmailAddress) { + headers += `From: ${msg.senderName} <${msg.senderEmailAddress}>\n`; + } + if (msg.displayTo) { + headers += `To: ${msg.displayTo}\n`; + } + if (msg.displayCC) { + headers += `Cc: ${msg.displayCC}\n`; + } + if (msg.displayBCC) { + headers += `Bcc: ${msg.displayBCC}\n`; + } + if (msg.subject) { + headers += `Subject: ${msg.subject}\n`; + } + if (msg.clientSubmitTime) { + headers += `Date: ${new Date(msg.clientSubmitTime).toUTCString()}\n`; + } + if (msg.internetMessageId) { + headers += `Message-ID: <${msg.internetMessageId}>\n`; + } + if (msg.inReplyToId) { + headers += `In-Reply-To: ${msg.inReplyToId}`; + } + if (msg.conversationId) { + headers += `Conversation-Id: ${msg.conversationId}`; + } + headers += 'MIME-Version: 1.0\n'; - if (msg.senderName || msg.senderEmailAddress) { - headers += `From: ${msg.senderName} <${msg.senderEmailAddress}>\n`; - } - if (msg.displayTo) { - headers += `To: ${msg.displayTo}\n`; - } - if (msg.displayCC) { - headers += `Cc: ${msg.displayCC}\n`; - } - if (msg.displayBCC) { - headers += `Bcc: ${msg.displayBCC}\n`; - } - if (msg.subject) { - headers += `Subject: ${msg.subject}\n`; - } - if (msg.clientSubmitTime) { - headers += `Date: ${new Date(msg.clientSubmitTime).toUTCString()}\n`; - } - if (msg.internetMessageId) { - headers += `Message-ID: <${msg.internetMessageId}>\n`; - } - if (msg.inReplyToId) { - headers += `In-Reply-To: ${msg.inReplyToId}`; - } - if (msg.conversationId) { - headers += `Conversation-Id: ${msg.conversationId}`; - } - headers += 'MIME-Version: 1.0\n'; + //add new headers + if (!/Content-Type:/i.test(headers)) { + if (msg.hasAttachments) { + headers += `Content-Type: multipart/mixed; boundary="${boundary}"\n`; + headers += `Content-Type: multipart/alternative; boundary="${altBoundary}"\n\n`; + eml += headers; + eml += `--${boundary}\n\n`; + } else { + eml += headers; + eml += `Content-Type: multipart/alternative; boundary="${altBoundary}"\n\n`; + } + } + // Body + const hasBody = !!msg.body; + const hasHtml = !!msg.bodyHTML; - //add new headers - if (!/Content-Type:/i.test(headers)) { - if (msg.hasAttachments) { - headers += `Content-Type: multipart/mixed; boundary="${boundary}"\n`; - headers += `Content-Type: multipart/alternative; boundary="${altBoundary}"\n\n`; - eml += headers; - eml += `--${boundary}\n\n`; - } else { - eml += headers; - eml += `Content-Type: multipart/alternative; boundary="${altBoundary}"\n\n`; - } - } - // Body - const hasBody = !!msg.body; - const hasHtml = !!msg.bodyHTML; + if (hasBody) { + eml += `--${altBoundary}\n`; + eml += 'Content-Type: text/plain; charset="utf-8"\n\n'; + eml += `${msg.body}\n\n`; + } - if (hasBody) { - eml += `--${altBoundary}\n`; - eml += 'Content-Type: text/plain; charset="utf-8"\n\n'; - eml += `${msg.body}\n\n`; - } + if (hasHtml) { + eml += `--${altBoundary}\n`; + eml += 'Content-Type: text/html; charset="utf-8"\n\n'; + eml += `${msg.bodyHTML}\n\n`; + } - if (hasHtml) { - eml += `--${altBoundary}\n`; - eml += 'Content-Type: text/html; charset="utf-8"\n\n'; - eml += `${msg.bodyHTML}\n\n`; - } + if (hasBody || hasHtml) { + eml += `--${altBoundary}--\n`; + } - if (hasBody || hasHtml) { - eml += `--${altBoundary}--\n`; - } + if (msg.hasAttachments) { + for (let i = 0; i < msg.numberOfAttachments; i++) { + const attachment = msg.getAttachment(i); + const attachmentStream = attachment.fileInputStream; + if (attachmentStream) { + const attachmentBuffer = Buffer.alloc(attachment.filesize); + attachmentStream.readCompletely(attachmentBuffer); + eml += `\n--${boundary}\n`; + eml += `Content-Type: ${attachment.mimeTag}; name="${attachment.longFilename}"\n`; + eml += `Content-Disposition: attachment; filename="${attachment.longFilename}"\n`; + eml += 'Content-Transfer-Encoding: base64\n\n'; + eml += `${attachmentBuffer.toString('base64')}\n`; + } + } + eml += `\n--${boundary}--`; + } - if (msg.hasAttachments) { - for (let i = 0; i < msg.numberOfAttachments; i++) { - const attachment = msg.getAttachment(i); - const attachmentStream = attachment.fileInputStream; - if (attachmentStream) { - const attachmentBuffer = Buffer.alloc(attachment.filesize); - attachmentStream.readCompletely(attachmentBuffer); - eml += `\n--${boundary}\n`; - eml += `Content-Type: ${attachment.mimeTag}; name="${attachment.longFilename}"\n`; - eml += `Content-Disposition: attachment; filename="${attachment.longFilename}"\n`; - eml += 'Content-Transfer-Encoding: base64\n\n'; - eml += `${attachmentBuffer.toString('base64')}\n`; - } - } - eml += `\n--${boundary}--`; - } + return eml; + } - return eml; - } - - public getUpdatedSyncState(): SyncState { - return {}; - } + public getUpdatedSyncState(): SyncState { + return {}; + } } diff --git a/packages/backend/src/services/ingestion-connectors/helpers/utils.ts b/packages/backend/src/services/ingestion-connectors/helpers/utils.ts index 46776d4..708d015 100644 --- a/packages/backend/src/services/ingestion-connectors/helpers/utils.ts +++ b/packages/backend/src/services/ingestion-connectors/helpers/utils.ts @@ -1,56 +1,54 @@ - import type { Headers } from 'mailparser'; function getHeaderValue(header: any): string | undefined { - if (typeof header === 'string') { - return header; - } - if (Array.isArray(header)) { - return getHeaderValue(header[0]); - } - if (typeof header === 'object' && header !== null && 'value' in header) { - return getHeaderValue(header.value); - } - return undefined; + if (typeof header === 'string') { + return header; + } + if (Array.isArray(header)) { + return getHeaderValue(header[0]); + } + if (typeof header === 'object' && header !== null && 'value' in header) { + return getHeaderValue(header.value); + } + return undefined; } export function getThreadId(headers: Headers): string | undefined { + const referencesHeader = headers.get('references'); - const referencesHeader = headers.get('references'); + if (referencesHeader) { + const references = getHeaderValue(referencesHeader); + if (references) { + return references.split(' ')[0].trim(); + } + } - if (referencesHeader) { - const references = getHeaderValue(referencesHeader); - if (references) { - return references.split(' ')[0].trim(); - } - } + const inReplyToHeader = headers.get('in-reply-to'); - const inReplyToHeader = headers.get('in-reply-to'); + if (inReplyToHeader) { + const inReplyTo = getHeaderValue(inReplyToHeader); + if (inReplyTo) { + return inReplyTo.trim(); + } + } - if (inReplyToHeader) { - const inReplyTo = getHeaderValue(inReplyToHeader); - if (inReplyTo) { - return inReplyTo.trim(); - } - } + const conversationIdHeader = headers.get('conversation-id'); - const conversationIdHeader = headers.get('conversation-id'); + if (conversationIdHeader) { + const conversationId = getHeaderValue(conversationIdHeader); + if (conversationId) { + return conversationId.trim(); + } + } - if (conversationIdHeader) { - const conversationId = getHeaderValue(conversationIdHeader); - if (conversationId) { - return conversationId.trim(); - } - } + const messageIdHeader = headers.get('message-id'); - const messageIdHeader = headers.get('message-id'); - - if (messageIdHeader) { - const messageId = getHeaderValue(messageIdHeader); - if (messageId) { - return messageId.trim(); - } - } - console.warn('No thread ID found, returning undefined'); - return undefined; + if (messageIdHeader) { + const messageId = getHeaderValue(messageIdHeader); + if (messageId) { + return messageId.trim(); + } + } + console.warn('No thread ID found, returning undefined'); + return undefined; } diff --git a/packages/backend/src/services/storage/LocalFileSystemProvider.ts b/packages/backend/src/services/storage/LocalFileSystemProvider.ts index 0e7c489..1c6138a 100644 --- a/packages/backend/src/services/storage/LocalFileSystemProvider.ts +++ b/packages/backend/src/services/storage/LocalFileSystemProvider.ts @@ -5,54 +5,54 @@ import { createReadStream, createWriteStream } from 'fs'; import { pipeline } from 'stream/promises'; export class LocalFileSystemProvider implements IStorageProvider { - private readonly rootPath: string; + private readonly rootPath: string; - constructor(config: LocalStorageConfig) { - this.rootPath = config.rootPath; - } + constructor(config: LocalStorageConfig) { + this.rootPath = config.rootPath; + } - async put(filePath: string, content: Buffer | NodeJS.ReadableStream): Promise { - const fullPath = path.join(this.rootPath, filePath); - const dir = path.dirname(fullPath); - await fs.mkdir(dir, { recursive: true }); + async put(filePath: string, content: Buffer | NodeJS.ReadableStream): Promise { + const fullPath = path.join(this.rootPath, filePath); + const dir = path.dirname(fullPath); + await fs.mkdir(dir, { recursive: true }); - if (Buffer.isBuffer(content)) { - await fs.writeFile(fullPath, content); - } else { - const writeStream = createWriteStream(fullPath); - await pipeline(content, writeStream); - } - } + if (Buffer.isBuffer(content)) { + await fs.writeFile(fullPath, content); + } else { + const writeStream = createWriteStream(fullPath); + await pipeline(content, writeStream); + } + } - async get(filePath: string): Promise { - const fullPath = path.join(this.rootPath, filePath); - try { - await fs.access(fullPath); - return createReadStream(fullPath); - } catch (error) { - throw new Error('File not found'); - } - } + async get(filePath: string): Promise { + const fullPath = path.join(this.rootPath, filePath); + try { + await fs.access(fullPath); + return createReadStream(fullPath); + } catch (error) { + throw new Error('File not found'); + } + } - async delete(filePath: string): Promise { - const fullPath = path.join(this.rootPath, filePath); - try { - await fs.rm(fullPath, { recursive: true, force: true }); - } catch (error: any) { - // Even with force: true, other errors might occur (e.g., permissions) - if (error.code !== 'ENOENT') { - throw error; - } - } - } + async delete(filePath: string): Promise { + const fullPath = path.join(this.rootPath, filePath); + try { + await fs.rm(fullPath, { recursive: true, force: true }); + } catch (error: any) { + // Even with force: true, other errors might occur (e.g., permissions) + if (error.code !== 'ENOENT') { + throw error; + } + } + } - async exists(filePath: string): Promise { - const fullPath = path.join(this.rootPath, filePath); - try { - await fs.access(fullPath); - return true; - } catch { - return false; - } - } + async exists(filePath: string): Promise { + const fullPath = path.join(this.rootPath, filePath); + try { + await fs.access(fullPath); + return true; + } catch { + return false; + } + } } diff --git a/packages/backend/src/services/storage/S3StorageProvider.ts b/packages/backend/src/services/storage/S3StorageProvider.ts index 6ecd53f..9f0736d 100644 --- a/packages/backend/src/services/storage/S3StorageProvider.ts +++ b/packages/backend/src/services/storage/S3StorageProvider.ts @@ -1,105 +1,105 @@ import { IStorageProvider, S3StorageConfig } from '@open-archiver/types'; import { - S3Client, - GetObjectCommand, - DeleteObjectCommand, - HeadObjectCommand, - NotFound, - ListObjectsV2Command, - DeleteObjectsCommand, + S3Client, + GetObjectCommand, + DeleteObjectCommand, + HeadObjectCommand, + NotFound, + ListObjectsV2Command, + DeleteObjectsCommand, } from '@aws-sdk/client-s3'; import { Upload } from '@aws-sdk/lib-storage'; import { Readable } from 'stream'; export class S3StorageProvider implements IStorageProvider { - private readonly client: S3Client; - private readonly bucket: string; + private readonly client: S3Client; + private readonly bucket: string; - constructor(config: S3StorageConfig) { - this.client = new S3Client({ - endpoint: config.endpoint, - region: config.region, - credentials: { - accessKeyId: config.accessKeyId, - secretAccessKey: config.secretAccessKey, - }, - forcePathStyle: config.forcePathStyle, - }); - this.bucket = config.bucket; - } + constructor(config: S3StorageConfig) { + this.client = new S3Client({ + endpoint: config.endpoint, + region: config.region, + credentials: { + accessKeyId: config.accessKeyId, + secretAccessKey: config.secretAccessKey, + }, + forcePathStyle: config.forcePathStyle, + }); + this.bucket = config.bucket; + } - async put(path: string, content: Buffer | NodeJS.ReadableStream): Promise { - const upload = new Upload({ - client: this.client, - params: { - Bucket: this.bucket, - Key: path, - Body: content instanceof Readable ? content : Readable.from(content), - }, - }); + async put(path: string, content: Buffer | NodeJS.ReadableStream): Promise { + const upload = new Upload({ + client: this.client, + params: { + Bucket: this.bucket, + Key: path, + Body: content instanceof Readable ? content : Readable.from(content), + }, + }); - await upload.done(); - } + await upload.done(); + } - async get(path: string): Promise { - const command = new GetObjectCommand({ - Bucket: this.bucket, - Key: path, - }); + async get(path: string): Promise { + const command = new GetObjectCommand({ + Bucket: this.bucket, + Key: path, + }); - try { - const response = await this.client.send(command); - if (response.Body instanceof Readable) { - return response.Body; - } - throw new Error('Readable stream not found in S3 response'); - } catch (error) { - if (error instanceof NotFound) { - throw new Error('File not found'); - } - throw error; - } - } + try { + const response = await this.client.send(command); + if (response.Body instanceof Readable) { + return response.Body; + } + throw new Error('Readable stream not found in S3 response'); + } catch (error) { + if (error instanceof NotFound) { + throw new Error('File not found'); + } + throw error; + } + } - async delete(path: string): Promise { - // List all objects with the given prefix - const listCommand = new ListObjectsV2Command({ - Bucket: this.bucket, - Prefix: path, - }); - const listedObjects = await this.client.send(listCommand); + async delete(path: string): Promise { + // List all objects with the given prefix + const listCommand = new ListObjectsV2Command({ + Bucket: this.bucket, + Prefix: path, + }); + const listedObjects = await this.client.send(listCommand); - if (!listedObjects.Contents || listedObjects.Contents.length === 0) { - return; - } + if (!listedObjects.Contents || listedObjects.Contents.length === 0) { + return; + } - // Create a list of objects to delete - const deleteParams = { - Bucket: this.bucket, - Delete: { - Objects: listedObjects.Contents.map(({ Key }) => ({ Key })), - }, - }; + // Create a list of objects to delete + const deleteParams = { + Bucket: this.bucket, + Delete: { + Objects: listedObjects.Contents.map(({ Key }) => ({ Key })), + }, + }; - // Delete the objects - const deleteCommand = new DeleteObjectsCommand(deleteParams); - await this.client.send(deleteCommand); - } + // Delete the objects + const deleteCommand = new DeleteObjectsCommand(deleteParams); + await this.client.send(deleteCommand); + } - async exists(path: string): Promise { - const command = new HeadObjectCommand({ - Bucket: this.bucket, - Key: path, - }); + async exists(path: string): Promise { + const command = new HeadObjectCommand({ + Bucket: this.bucket, + Key: path, + }); - try { - await this.client.send(command); - return true; - } catch (error) { - if (error instanceof NotFound) { - return false; - } - throw error; - } - } + try { + await this.client.send(command); + return true; + } catch (error) { + if (error instanceof NotFound) { + return false; + } + throw error; + } + } } diff --git a/packages/backend/src/workers/indexing.worker.ts b/packages/backend/src/workers/indexing.worker.ts index d38c471..56686b9 100644 --- a/packages/backend/src/workers/indexing.worker.ts +++ b/packages/backend/src/workers/indexing.worker.ts @@ -3,23 +3,23 @@ import { connection } from '../config/redis'; import indexEmailProcessor from '../jobs/processors/index-email.processor'; const processor = async (job: any) => { - switch (job.name) { - case 'index-email': - return indexEmailProcessor(job); - default: - throw new Error(`Unknown job name: ${job.name}`); - } + switch (job.name) { + case 'index-email': + return indexEmailProcessor(job); + default: + throw new Error(`Unknown job name: ${job.name}`); + } }; const worker = new Worker('indexing', processor, { - connection, - concurrency: 5, - removeOnComplete: { - count: 1000, // keep last 1000 jobs - }, - removeOnFail: { - count: 5000, // keep last 5000 failed jobs - }, + connection, + concurrency: 5, + removeOnComplete: { + count: 1000, // keep last 1000 jobs + }, + removeOnFail: { + count: 5000, // keep last 5000 failed jobs + }, }); console.log('Indexing worker started'); diff --git a/packages/backend/src/workers/ingestion.worker.ts b/packages/backend/src/workers/ingestion.worker.ts index bfc26ef..8b9ff6a 100644 --- a/packages/backend/src/workers/ingestion.worker.ts +++ b/packages/backend/src/workers/ingestion.worker.ts @@ -7,33 +7,32 @@ import { processMailboxProcessor } from '../jobs/processors/process-mailbox.proc import syncCycleFinishedProcessor from '../jobs/processors/sync-cycle-finished.processor'; const processor = async (job: any) => { - switch (job.name) { - case 'initial-import': - return initialImportProcessor(job); - case 'sync-cycle-finished': - return syncCycleFinishedProcessor(job); - case 'continuous-sync': - return continuousSyncProcessor(job); - case 'schedule-continuous-sync': - return scheduleContinuousSyncProcessor(job); - case 'process-mailbox': - return processMailboxProcessor(job); - default: - throw new Error(`Unknown job name: ${job.name}`); - } + switch (job.name) { + case 'initial-import': + return initialImportProcessor(job); + case 'sync-cycle-finished': + return syncCycleFinishedProcessor(job); + case 'continuous-sync': + return continuousSyncProcessor(job); + case 'schedule-continuous-sync': + return scheduleContinuousSyncProcessor(job); + case 'process-mailbox': + return processMailboxProcessor(job); + default: + throw new Error(`Unknown job name: ${job.name}`); + } }; const worker = new Worker('ingestion', processor, { - connection, - removeOnComplete: { - count: 100, // keep last 100 jobs - }, - removeOnFail: { - count: 500, // keep last 500 failed jobs - }, + connection, + removeOnComplete: { + count: 100, // keep last 100 jobs + }, + removeOnFail: { + count: 500, // keep last 500 failed jobs + }, }); - console.log('Ingestion worker started'); process.on('SIGINT', () => worker.close()); diff --git a/packages/backend/tsconfig.json b/packages/backend/tsconfig.json index 06ef33b..39366b9 100644 --- a/packages/backend/tsconfig.json +++ b/packages/backend/tsconfig.json @@ -1,11 +1,11 @@ { - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "outDir": "./dist", - "rootDir": "./src", - "emitDecoratorMetadata": true, - "experimentalDecorators": true - }, - "include": ["src/**/*.ts"], - "exclude": ["node_modules", "dist"] + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "emitDecoratorMetadata": true, + "experimentalDecorators": true + }, + "include": ["src/**/*.ts"], + "exclude": ["node_modules", "dist"] } diff --git a/packages/frontend/src/app.d.ts b/packages/frontend/src/app.d.ts index 8eadfbd..2190b46 100644 --- a/packages/frontend/src/app.d.ts +++ b/packages/frontend/src/app.d.ts @@ -15,4 +15,4 @@ declare global { } } -export { }; +export {}; diff --git a/packages/frontend/src/hooks.server.ts b/packages/frontend/src/hooks.server.ts index 2c5d799..0322141 100644 --- a/packages/frontend/src/hooks.server.ts +++ b/packages/frontend/src/hooks.server.ts @@ -3,27 +3,25 @@ import { jwtVerify } from 'jose'; import type { User } from '@open-archiver/types'; import 'dotenv/config'; - - const JWT_SECRET_ENCODED = new TextEncoder().encode(process.env.JWT_SECRET); export const handle: Handle = async ({ event, resolve }) => { - const token = event.cookies.get('accessToken'); + const token = event.cookies.get('accessToken'); - if (token) { - try { - const { payload } = await jwtVerify(token, JWT_SECRET_ENCODED); - event.locals.user = payload as Omit; - event.locals.accessToken = token; - } catch (error) { - console.error('JWT verification failed:', error); - event.locals.user = null; - event.locals.accessToken = null; - } - } else { - event.locals.user = null; - event.locals.accessToken = null; - } + if (token) { + try { + const { payload } = await jwtVerify(token, JWT_SECRET_ENCODED); + event.locals.user = payload as Omit; + event.locals.accessToken = token; + } catch (error) { + console.error('JWT verification failed:', error); + event.locals.user = null; + event.locals.accessToken = null; + } + } else { + event.locals.user = null; + event.locals.accessToken = null; + } - return resolve(event); + return resolve(event); }; diff --git a/packages/frontend/src/lib/api.client.ts b/packages/frontend/src/lib/api.client.ts index d8732ea..c3f189e 100644 --- a/packages/frontend/src/lib/api.client.ts +++ b/packages/frontend/src/lib/api.client.ts @@ -9,28 +9,25 @@ const BASE_URL = '/api/v1'; // Using a relative URL for proxying * @param options The standard Fetch API options. * @returns A Promise that resolves to the Fetch Response. */ -export const api = async ( - url: string, - options: RequestInit = {} -): Promise => { - const { accessToken } = get(authStore); - const defaultHeaders: HeadersInit = {}; +export const api = async (url: string, options: RequestInit = {}): Promise => { + const { accessToken } = get(authStore); + const defaultHeaders: HeadersInit = {}; - if (!(options.body instanceof FormData)) { - defaultHeaders['Content-Type'] = 'application/json'; - } + if (!(options.body instanceof FormData)) { + defaultHeaders['Content-Type'] = 'application/json'; + } - if (accessToken) { - defaultHeaders['Authorization'] = `Bearer ${accessToken}`; - } + if (accessToken) { + defaultHeaders['Authorization'] = `Bearer ${accessToken}`; + } - const mergedOptions: RequestInit = { - ...options, - headers: { - ...defaultHeaders, - ...options.headers - } - }; + const mergedOptions: RequestInit = { + ...options, + headers: { + ...defaultHeaders, + ...options.headers, + }, + }; - return fetch(`${BASE_URL}${url}`, mergedOptions); + return fetch(`${BASE_URL}${url}`, mergedOptions); }; diff --git a/packages/frontend/src/lib/components/custom/EmailPreview.svelte b/packages/frontend/src/lib/components/custom/EmailPreview.svelte index dc3960f..d28aacf 100644 --- a/packages/frontend/src/lib/components/custom/EmailPreview.svelte +++ b/packages/frontend/src/lib/components/custom/EmailPreview.svelte @@ -4,8 +4,9 @@ let { raw, - rawHtml - }: { raw?: Buffer | { type: 'Buffer'; data: number[] } | undefined; rawHtml?: string } = $props(); + rawHtml, + }: { raw?: Buffer | { type: 'Buffer'; data: number[] } | undefined; rawHtml?: string } = + $props(); let parsedEmail: Email | null = $state(null); let isLoading = $state(true); diff --git a/packages/frontend/src/lib/components/custom/EmailThread.svelte b/packages/frontend/src/lib/components/custom/EmailThread.svelte index 18d5b3f..9756d67 100644 --- a/packages/frontend/src/lib/components/custom/EmailThread.svelte +++ b/packages/frontend/src/lib/components/custom/EmailThread.svelte @@ -5,7 +5,7 @@ let { thread, - currentEmailId + currentEmailId, }: { thread: ArchivedEmail['thread']; currentEmailId: string; @@ -45,7 +45,7 @@ onclick={(e) => { e.preventDefault(); goto(`/dashboard/archived-emails/${item.id}`, { - invalidateAll: true + invalidateAll: true, }); }}>{item.subject || 'No Subject'} @@ -53,7 +53,9 @@ {item.subject || 'No Subject'} {/if} -
+
From: {item.senderEmail}
diff --git a/packages/frontend/src/lib/components/custom/EmptyState.svelte b/packages/frontend/src/lib/components/custom/EmptyState.svelte index 5c23b60..684f6e1 100644 --- a/packages/frontend/src/lib/components/custom/EmptyState.svelte +++ b/packages/frontend/src/lib/components/custom/EmptyState.svelte @@ -5,7 +5,7 @@ header, text, buttonText, - click + click, }: { header: string; text: string; diff --git a/packages/frontend/src/lib/components/custom/Footer.svelte b/packages/frontend/src/lib/components/custom/Footer.svelte index 1226b7f..150d7e2 100644 --- a/packages/frontend/src/lib/components/custom/Footer.svelte +++ b/packages/frontend/src/lib/components/custom/Footer.svelte @@ -5,7 +5,8 @@

© {new Date().getFullYear()} - Open Archiver. All rights reserved. + Open Archiver. All rights + reserved.

diff --git a/packages/frontend/src/lib/components/custom/IngestionSourceForm.svelte b/packages/frontend/src/lib/components/custom/IngestionSourceForm.svelte index 79b8704..37201f7 100644 --- a/packages/frontend/src/lib/components/custom/IngestionSourceForm.svelte +++ b/packages/frontend/src/lib/components/custom/IngestionSourceForm.svelte @@ -13,7 +13,7 @@ import { Loader2 } from 'lucide-svelte'; let { source = null, - onSubmit + onSubmit, }: { source?: IngestionSource | null; onSubmit: (data: CreateIngestionSourceDto) => Promise; @@ -24,7 +24,7 @@ { value: 'google_workspace', label: 'Google Workspace' }, { value: 'microsoft_365', label: 'Microsoft 365' }, { value: 'pst_import', label: 'PST Import' }, - { value: 'eml_import', label: 'EML Import' } + { value: 'eml_import', label: 'EML Import' }, ]; let formData: CreateIngestionSourceDto = $state({ @@ -32,8 +32,8 @@ provider: source?.provider ?? 'generic_imap', providerConfig: source?.credentials ?? { type: source?.provider ?? 'generic_imap', - secure: true - } + secure: true, + }, }); $effect(() => { @@ -74,7 +74,7 @@ try { const response = await api('/upload', { method: 'POST', - body: uploadFormData + body: uploadFormData, }); const result = await response.json(); if (!response.ok) { @@ -92,7 +92,7 @@ title: 'Upload Failed, please try again', message: JSON.stringify(error), duration: 5000, - show: true + show: true, }); } }; @@ -161,7 +161,12 @@
- +
@@ -184,7 +189,13 @@
- + {#if fileUploading} {/if} @@ -194,7 +205,13 @@
- + {#if fileUploading} {/if} @@ -206,9 +223,9 @@ Heads up!
- Please note that this is an organization-wide operation. This kind of ingestions will - import and index all email inboxes in your organization. If you want to import only - specific email inboxes, use the IMAP connector. + Please note that this is an organization-wide operation. This kind of ingestions + will import and index all email inboxes in your organization. If you want + to import only specific email inboxes, use the IMAP connector.
diff --git a/packages/frontend/src/lib/components/custom/alert/Alerts.svelte b/packages/frontend/src/lib/components/custom/alert/Alerts.svelte index 9acce51..bbe7c12 100644 --- a/packages/frontend/src/lib/components/custom/alert/Alerts.svelte +++ b/packages/frontend/src/lib/components/custom/alert/Alerts.svelte @@ -15,7 +15,7 @@ icon: 'heroicons-outline:check-circle', color: 'text-green-800', messageColor: 'text-green-700', - bgColor: 'text-green-50' + bgColor: 'text-green-50', }); $effect(() => { show; @@ -30,21 +30,21 @@ icon: 'heroicons-outline:check-circle', color: 'text-green-600', messageColor: 'text-green-500', - bgColor: 'bg-green-50' + bgColor: 'bg-green-50', }; } else if (type === 'error') { styleConfig = { icon: 'heroicons-outline:exclamation-circle', color: 'text-yellow-600', messageColor: 'text-yellow-600', - bgColor: 'bg-yellow-50' + bgColor: 'bg-yellow-50', }; } else if (type === 'warning') { styleConfig = { icon: 'heroicons-outline:exclamation', color: 'text-yellow-600', messageColor: 'text-yellow-600', - bgColor: 'bg-yellow-50' + bgColor: 'bg-yellow-50', }; } }); @@ -59,7 +59,7 @@ {#if show} + {/if} +
+
- - - - Are you sure you want to delete this email? - - This action cannot be undone and will permanently remove the email and its - attachments. - - - - - - - - - - + + + + Are you sure you want to delete this email? + + This action cannot be undone and will permanently remove the email and its + attachments. + + + + + + + + + + {:else} -

Email not found.

+

Email not found.

{/if} diff --git a/packages/frontend/src/routes/dashboard/ingestions/+page.server.ts b/packages/frontend/src/routes/dashboard/ingestions/+page.server.ts index 5ca00a6..0288719 100644 --- a/packages/frontend/src/routes/dashboard/ingestions/+page.server.ts +++ b/packages/frontend/src/routes/dashboard/ingestions/+page.server.ts @@ -3,20 +3,20 @@ import type { PageServerLoad } from './$types'; import type { IngestionSource } from '@open-archiver/types'; export const load: PageServerLoad = async (event) => { - try { - const response = await api('/ingestion-sources', event); - if (!response.ok) { - throw new Error(`Failed to fetch ingestion sources: ${response.statusText}`); - } - const ingestionSources: IngestionSource[] = await response.json(); - return { - ingestionSources - }; - } catch (error) { - console.error('Failed to load ingestion sources:', error); - return { - ingestionSources: [], - error: 'Failed to load ingestion sources' - }; - } + try { + const response = await api('/ingestion-sources', event); + if (!response.ok) { + throw new Error(`Failed to fetch ingestion sources: ${response.statusText}`); + } + const ingestionSources: IngestionSource[] = await response.json(); + return { + ingestionSources, + }; + } catch (error) { + console.error('Failed to load ingestion sources:', error); + return { + ingestionSources: [], + error: 'Failed to load ingestion sources', + }; + } }; diff --git a/packages/frontend/src/routes/dashboard/ingestions/+page.svelte b/packages/frontend/src/routes/dashboard/ingestions/+page.svelte index 5b51c9c..058a72b 100644 --- a/packages/frontend/src/routes/dashboard/ingestions/+page.svelte +++ b/packages/frontend/src/routes/dashboard/ingestions/+page.svelte @@ -36,7 +36,7 @@ title: 'Demo mode', message: 'Editing is not allowed in demo mode.', duration: 5000, - show: true + show: true, }); return; } @@ -61,7 +61,7 @@ title: 'Failed to delete ingestion', message: errorBody.message || JSON.stringify(errorBody), duration: 5000, - show: true + show: true, }); return; } @@ -82,7 +82,7 @@ title: 'Failed to trigger force sync ingestion', message: errorBody.message || JSON.stringify(errorBody), duration: 5000, - show: true + show: true, }); return; } @@ -107,7 +107,7 @@ } else { await api(`/ingestion-sources/${source.id}`, { method: 'PUT', - body: JSON.stringify({ status: 'active' }) + body: JSON.stringify({ status: 'active' }), }); } @@ -123,7 +123,7 @@ title: 'Failed to trigger force sync ingestion', message: e instanceof Error ? e.message : JSON.stringify(e), duration: 5000, - show: true + show: true, }); } }; @@ -140,7 +140,7 @@ title: `Failed to delete ingestion ${id}`, message: errorBody.message || JSON.stringify(errorBody), duration: 5000, - show: true + show: true, }); } } @@ -163,7 +163,7 @@ title: `Failed to trigger force sync for ingestion ${id}`, message: errorBody.message || JSON.stringify(errorBody), duration: 5000, - show: true + show: true, }); } } @@ -181,7 +181,7 @@ title: 'Failed to trigger force sync', message: e instanceof Error ? e.message : JSON.stringify(e), duration: 5000, - show: true + show: true, }); } }; @@ -192,7 +192,7 @@ // Update const response = await api(`/ingestion-sources/${selectedSource.id}`, { method: 'PUT', - body: JSON.stringify(formData) + body: JSON.stringify(formData), }); if (!response.ok) { const errorData = await response.json(); @@ -206,7 +206,7 @@ // Create const response = await api('/ingestion-sources', { method: 'POST', - body: JSON.stringify(formData) + body: JSON.stringify(formData), }); if (!response.ok) { const errorData = await response.json(); @@ -226,7 +226,7 @@ title: 'Authentication Failed', message, duration: 5000, - show: true + show: true, }); } }; @@ -276,7 +276,10 @@ Force Sync - (isBulkDeleteDialogOpen = true)}> + (isBulkDeleteDialogOpen = true)} + > Delete @@ -300,7 +303,8 @@ selectedIds = []; } }} - checked={ingestionSources.length > 0 && selectedIds.length === ingestionSources.length + checked={ingestionSources.length > 0 && + selectedIds.length === ingestionSources.length ? true : ((selectedIds.length > 0 ? 'indeterminate' : false) as any)} /> @@ -322,7 +326,9 @@ checked={selectedIds.includes(source.id)} onCheckedChange={() => { if (selectedIds.includes(source.id)) { - selectedIds = selectedIds.filter((id) => id !== source.id); + selectedIds = selectedIds.filter( + (id) => id !== source.id + ); } else { selectedIds = [...selectedIds, source.id]; } @@ -330,15 +336,23 @@ /> - {source.name} - {source.provider.split('_').join(' ')} + {source.provider.split('_').join(' ')} - + {source.status.split('_').join(' ')} @@ -358,10 +372,13 @@ class="cursor-pointer" checked={source.status !== 'paused'} onCheckedChange={() => handleToggle(source)} - disabled={source.status === 'importing' || source.status === 'syncing'} + disabled={source.status === 'importing' || + source.status === 'syncing'} /> - {new Date(source.createdAt).toLocaleDateString()} + {new Date(source.createdAt).toLocaleDateString()} @@ -379,7 +396,9 @@ >Force sync - openDeleteDialog(source)} + openDeleteDialog(source)} >Delete @@ -409,7 +428,8 @@ >Read docs heredocs here. @@ -423,12 +443,17 @@ Are you sure you want to delete this ingestion? - This will delete all archived emails, attachments, indexing, and files associated with this - ingestion. If you only want to stop syncing new emails, you can pause the ingestion instead. + This will delete all archived emails, attachments, indexing, and files associated + with this ingestion. If you only want to stop syncing new emails, you can pause the + ingestion instead. - @@ -445,13 +470,17 @@ >Are you sure you want to delete {selectedIds.length} selected ingestions? - This will delete all archived emails, attachments, indexing, and files associated with these - ingestions. If you only want to stop syncing new emails, you can pause the ingestions - instead. + This will delete all archived emails, attachments, indexing, and files associated + with these ingestions. If you only want to stop syncing new emails, you can pause + the ingestions instead. - diff --git a/packages/frontend/src/routes/dashboard/search/+page.server.ts b/packages/frontend/src/routes/dashboard/search/+page.server.ts index cda6ac1..552a9ff 100644 --- a/packages/frontend/src/routes/dashboard/search/+page.server.ts +++ b/packages/frontend/src/routes/dashboard/search/+page.server.ts @@ -5,46 +5,46 @@ import type { SearchResult } from '@open-archiver/types'; import type { MatchingStrategy } from '@open-archiver/types'; async function performSearch( - keywords: string, - page: number, - matchingStrategy: MatchingStrategy, - event: RequestEvent + keywords: string, + page: number, + matchingStrategy: MatchingStrategy, + event: RequestEvent ) { - if (!keywords) { - return { searchResult: null, keywords: '', page: 1, matchingStrategy: 'last' }; - } + if (!keywords) { + return { searchResult: null, keywords: '', page: 1, matchingStrategy: 'last' }; + } - try { - const response = await api( - `/search?keywords=${keywords}&page=${page}&limit=10&matchingStrategy=${matchingStrategy}`, - event, - { - method: 'GET' - } - ); + try { + const response = await api( + `/search?keywords=${keywords}&page=${page}&limit=10&matchingStrategy=${matchingStrategy}`, + event, + { + method: 'GET', + } + ); - if (!response.ok) { - const error = await response.json(); - return { searchResult: null, keywords, page, matchingStrategy, error: error.message }; - } + if (!response.ok) { + const error = await response.json(); + return { searchResult: null, keywords, page, matchingStrategy, error: error.message }; + } - const searchResult = (await response.json()) as SearchResult; - return { searchResult, keywords, page, matchingStrategy }; - } catch (error) { - return { - searchResult: null, - keywords, - page, - matchingStrategy, - error: error instanceof Error ? error.message : 'Unknown error' - }; - } + const searchResult = (await response.json()) as SearchResult; + return { searchResult, keywords, page, matchingStrategy }; + } catch (error) { + return { + searchResult: null, + keywords, + page, + matchingStrategy, + error: error instanceof Error ? error.message : 'Unknown error', + }; + } } export const load: PageServerLoad = async (event) => { - const keywords = event.url.searchParams.get('keywords') || ''; - const page = parseInt(event.url.searchParams.get('page') || '1'); - const matchingStrategy = (event.url.searchParams.get('matchingStrategy') || - 'last') as MatchingStrategy; - return performSearch(keywords, page, matchingStrategy, event); + const keywords = event.url.searchParams.get('keywords') || ''; + const page = parseInt(event.url.searchParams.get('page') || '1'); + const matchingStrategy = (event.url.searchParams.get('matchingStrategy') || + 'last') as MatchingStrategy; + return performSearch(keywords, page, matchingStrategy, event); }; diff --git a/packages/frontend/src/routes/dashboard/search/+page.svelte b/packages/frontend/src/routes/dashboard/search/+page.svelte index 67c80e0..2a96106 100644 --- a/packages/frontend/src/routes/dashboard/search/+page.svelte +++ b/packages/frontend/src/routes/dashboard/search/+page.svelte @@ -8,7 +8,7 @@ CardContent, CardHeader, CardTitle, - CardDescription + CardDescription, } from '$lib/components/ui/card'; import { onMount } from 'svelte'; import { goto } from '$app/navigation'; @@ -27,7 +27,7 @@ const strategies = [ { value: 'last', label: 'Fuzzy' }, { value: 'all', label: 'Verbatim' }, - { value: 'frequency', label: 'Frequency' } + { value: 'frequency', label: 'Frequency' }, ]; const triggerContent = $derived( @@ -54,7 +54,7 @@ update(newHtml: string | undefined) { if (newHtml === undefined) return; content.innerHTML = newHtml; - } + }, }; } @@ -160,7 +160,10 @@ }; let paginationItems = $derived( - getPaginationItems(page, Math.ceil((searchResult?.total || 0) / (searchResult?.limit || 10))) + getPaginationItems( + page, + Math.ceil((searchResult?.total || 0) / (searchResult?.limit || 10)) + ) ); @@ -191,7 +194,11 @@ {#each strategies as strategy (strategy.value)} - + {strategy.label} {/each} @@ -229,23 +236,30 @@ From: {#if !isMounted} - + {:else} - + {/if} | To: {#if !isMounted} - + {:else} {/if} | {#if !isMounted} - + {:else} {new Date(hit.timestamp).toLocaleString()} @@ -257,12 +271,17 @@ {#if _formatted.body} {#each getHighlightedSnippets(_formatted.body) as snippet} -
+

In email body:

{#if !isMounted} {:else} -

+

{/if}
{/each} @@ -273,14 +292,19 @@ {#each _formatted.attachments as attachment, i} {#if attachment && attachment.content} {#each getHighlightedSnippets(attachment.content) as snippet} -
+

In attachment: {attachment.filename}

{#if !isMounted} {:else} -

+

{/if}
{/each} @@ -326,7 +350,8 @@ > Next
diff --git a/packages/frontend/src/routes/setup/+page.server.ts b/packages/frontend/src/routes/setup/+page.server.ts index 8baf2b9..dedc2a8 100644 --- a/packages/frontend/src/routes/setup/+page.server.ts +++ b/packages/frontend/src/routes/setup/+page.server.ts @@ -1,5 +1,4 @@ import { redirect } from '@sveltejs/kit'; -import type { PageServerLoad } from "./$types"; +import type { PageServerLoad } from './$types'; - -export const load = (async (event) => { }) satisfies PageServerLoad; \ No newline at end of file +export const load = (async (event) => {}) satisfies PageServerLoad; diff --git a/packages/frontend/src/routes/setup/+page.svelte b/packages/frontend/src/routes/setup/+page.svelte index ab4fa15..511b3fe 100644 --- a/packages/frontend/src/routes/setup/+page.svelte +++ b/packages/frontend/src/routes/setup/+page.svelte @@ -19,7 +19,7 @@ try { const response = await api('/auth/setup', { method: 'POST', - body: JSON.stringify({ first_name, last_name, email, password }) + body: JSON.stringify({ first_name, last_name, email, password }), }); if (!response.ok) { @@ -36,7 +36,7 @@ title: 'Setup Failed', message: err.message, duration: 5000, - show: true + show: true, }); } finally { isLoading = false; @@ -46,7 +46,10 @@ Setup - Open Archiver - +
Welcome - Create the first administrator account to get started. + Create the first administrator account to get started.
@@ -91,7 +96,13 @@
- +
diff --git a/packages/frontend/src/routes/signin/+page.server.ts b/packages/frontend/src/routes/signin/+page.server.ts index f715ce1..dba2ac5 100644 --- a/packages/frontend/src/routes/signin/+page.server.ts +++ b/packages/frontend/src/routes/signin/+page.server.ts @@ -1,11 +1,9 @@ import { redirect } from '@sveltejs/kit'; -import type { PageServerLoad } from "./$types"; - +import type { PageServerLoad } from './$types'; export const load = (async (event) => { - const { locals } = event; - if (locals.user) { - throw redirect(307, '/dashboard'); - } - -}) satisfies PageServerLoad; \ No newline at end of file + const { locals } = event; + if (locals.user) { + throw redirect(307, '/dashboard'); + } +}) satisfies PageServerLoad; diff --git a/packages/frontend/src/routes/signin/+page.svelte b/packages/frontend/src/routes/signin/+page.svelte index 9458efa..699a6b5 100644 --- a/packages/frontend/src/routes/signin/+page.svelte +++ b/packages/frontend/src/routes/signin/+page.svelte @@ -18,7 +18,7 @@ try { const response = await api('/auth/login', { method: 'POST', - body: JSON.stringify({ email, password }) + body: JSON.stringify({ email, password }), }); if (!response.ok) { let errorMessage = 'Failed to login'; @@ -41,7 +41,7 @@ title: 'Login Failed', message: e.message, duration: 5000, - show: true + show: true, }); } finally { isLoading = false; @@ -76,7 +76,13 @@
- +
diff --git a/packages/frontend/svelte.config.js b/packages/frontend/svelte.config.js index 0dbd5e5..e2e33af 100644 --- a/packages/frontend/svelte.config.js +++ b/packages/frontend/svelte.config.js @@ -7,8 +7,8 @@ const config = { // for more information about preprocessors preprocess: vitePreprocess(), kit: { - adapter: adapter() - } + adapter: adapter(), + }, }; export default config; diff --git a/packages/frontend/vite.config.ts b/packages/frontend/vite.config.ts index d9bc45a..53d4b67 100644 --- a/packages/frontend/vite.config.ts +++ b/packages/frontend/vite.config.ts @@ -13,11 +13,11 @@ export default defineConfig({ '/api': { target: `http://localhost:${process.env.PORT_BACKEND || 4000}`, changeOrigin: true, - rewrite: (path) => path.replace(/^\/api/, '') - } - } + rewrite: (path) => path.replace(/^\/api/, ''), + }, + }, }, ssr: { - noExternal: ['layerchart'] - } + noExternal: ['layerchart'], + }, }); diff --git a/packages/types/package.json b/packages/types/package.json index 95a6cee..824cdfa 100644 --- a/packages/types/package.json +++ b/packages/types/package.json @@ -1,18 +1,18 @@ { - "name": "@open-archiver/types", - "version": "0.1.0", - "private": true, - "main": "dist/index.js", - "types": "dist/index.d.ts", - "scripts": { - "build": "tsc", - "dev": "tsc --watch" - }, - "devDependencies": { - "@types/node": "^24.0.13", - "typescript": "^5.0.0" - }, - "dependencies": { - "jose": "^6.0.11" - } + "name": "@open-archiver/types", + "version": "0.1.0", + "private": true, + "main": "dist/index.js", + "types": "dist/index.d.ts", + "scripts": { + "build": "tsc", + "dev": "tsc --watch" + }, + "devDependencies": { + "@types/node": "^24.0.13", + "typescript": "^5.0.0" + }, + "dependencies": { + "jose": "^6.0.11" + } } diff --git a/packages/types/src/archived-emails.types.ts b/packages/types/src/archived-emails.types.ts index 4f4a3f0..3a402d5 100644 --- a/packages/types/src/archived-emails.types.ts +++ b/packages/types/src/archived-emails.types.ts @@ -2,62 +2,61 @@ * Represents a single recipient of an email. */ export interface Recipient { - name?: string; - email: string; + name?: string; + email: string; } /** * Represents a single attachment of an email. */ export interface Attachment { - id: string; - filename: string; - mimeType: string | null; - sizeBytes: number; - storagePath: string; + id: string; + filename: string; + mimeType: string | null; + sizeBytes: number; + storagePath: string; } - export interface ThreadEmail { - id: string; //the archivedemail id - subject: string | null; - sentAt: Date; - senderEmail: string; + id: string; //the archivedemail id + subject: string | null; + sentAt: Date; + senderEmail: string; } /** * Represents a single archived email. */ export interface ArchivedEmail { - id: string; - ingestionSourceId: string; - userEmail: string; - messageIdHeader: string | null; - sentAt: Date; - subject: string | null; - senderName: string | null; - senderEmail: string; - recipients: Recipient[]; - storagePath: string; - storageHashSha256: string; - sizeBytes: number; - isIndexed: boolean; - hasAttachments: boolean; - isOnLegalHold: boolean; - archivedAt: Date; - attachments?: Attachment[]; - raw?: Buffer; - thread?: ThreadEmail[]; - path: string | null; - tags: string[] | null; + id: string; + ingestionSourceId: string; + userEmail: string; + messageIdHeader: string | null; + sentAt: Date; + subject: string | null; + senderName: string | null; + senderEmail: string; + recipients: Recipient[]; + storagePath: string; + storageHashSha256: string; + sizeBytes: number; + isIndexed: boolean; + hasAttachments: boolean; + isOnLegalHold: boolean; + archivedAt: Date; + attachments?: Attachment[]; + raw?: Buffer; + thread?: ThreadEmail[]; + path: string | null; + tags: string[] | null; } /** * Represents a paginated list of archived emails. */ export interface PaginatedArchivedEmails { - items: ArchivedEmail[]; - total: number; - page: number; - limit: number; + items: ArchivedEmail[]; + total: number; + page: number; + limit: number; } diff --git a/packages/types/src/auth.types.ts b/packages/types/src/auth.types.ts index 8322fbf..55455de 100644 --- a/packages/types/src/auth.types.ts +++ b/packages/types/src/auth.types.ts @@ -6,26 +6,26 @@ import type { User } from './user.types'; * This is the data that will be encoded into the token. */ export interface AuthTokenPayload extends JWTPayload { - /** - * The user's email address. - */ - email: string; - /** - * The user's assigned roles, which determines their permissions. - */ - roles: string[]; + /** + * The user's email address. + */ + email: string; + /** + * The user's assigned roles, which determines their permissions. + */ + roles: string[]; } /** * Defines the structure of the response from a successful login request. */ export interface LoginResponse { - /** - * The JSON Web Token for authenticating subsequent requests. - */ - accessToken: string; - /** - * The authenticated user's information. - */ - user: Omit; + /** + * The JSON Web Token for authenticating subsequent requests. + */ + accessToken: string; + /** + * The authenticated user's information. + */ + user: Omit; } diff --git a/packages/types/src/dashboard.types.ts b/packages/types/src/dashboard.types.ts index ee2b2b1..7276191 100644 --- a/packages/types/src/dashboard.types.ts +++ b/packages/types/src/dashboard.types.ts @@ -1,38 +1,38 @@ export interface DashboardStats { - totalEmailsArchived: number; - totalStorageUsed: number; - failedIngestionsLast7Days: number; + totalEmailsArchived: number; + totalStorageUsed: number; + failedIngestionsLast7Days: number; } export interface IngestionHistory { - history: { - date: string; - count: number; - }[]; + history: { + date: string; + count: number; + }[]; } export interface IngestionSourceStats { - id: string; - name: string; - provider: string; - status: string; - storageUsed: number; + id: string; + name: string; + provider: string; + status: string; + storageUsed: number; } export interface RecentSync { - id: string; - sourceName: string; - startTime: string; - duration: number; - emailsProcessed: number; - status: string; + id: string; + sourceName: string; + startTime: string; + duration: number; + emailsProcessed: number; + status: string; } export interface TopSender { - sender: string; - count: number; + sender: string; + count: number; } export interface IndexedInsights { - topSenders: TopSender[]; + topSenders: TopSender[]; } diff --git a/packages/types/src/email.types.ts b/packages/types/src/email.types.ts index 5e64322..83c0270 100644 --- a/packages/types/src/email.types.ts +++ b/packages/types/src/email.types.ts @@ -2,18 +2,18 @@ * Represents a single email address, including an optional name and the email address itself. */ export interface EmailAddress { - name: string; - address: string; + name: string; + address: string; } /** * Defines the structure for an email attachment, including its filename, content type, size, and the raw content as a buffer. */ export interface EmailAttachment { - filename: string; - contentType: string; - size: number; - content: Buffer; + filename: string; + contentType: string; + size: number; + content: Buffer; } /** @@ -21,54 +21,54 @@ export interface EmailAttachment { * This type serves as a standardized representation of an email before it is processed and stored in the database. */ export interface EmailObject { - /** A unique identifier for the email, typically assigned by the source provider. */ - id: string; - /** An optional identifier for the email thread, used to group related emails. */ - threadId?: string; - /** An array of `EmailAddress` objects representing the sender(s). */ - from: EmailAddress[]; - /** An array of `EmailAddress` objects representing the primary recipient(s). */ - to: EmailAddress[]; - /** An optional array of `EmailAddress` objects for carbon copy (CC) recipients. */ - cc?: EmailAddress[]; - /** An optional array of `EmailAddress` objects for blind carbon copy (BCC) recipients. */ - bcc?: EmailAddress[]; - /** The subject line of the email. */ - subject: string; - /** The plain text body of the email. */ - body: string; - /** The HTML version of the email body, if available. */ - html: string; - /** A map of all email headers, where keys are header names and values can be a string, an array of strings, or a complex object. */ - headers: Map; - /** An array of `EmailAttachment` objects found in the email. */ - attachments: EmailAttachment[]; - /** The date and time when the email was received. */ - receivedAt: Date; - /** An optional buffer containing the full raw EML content of the email, which is useful for archival and compliance purposes. */ - eml?: Buffer; - /** The email address of the user whose mailbox this email belongs to. */ - userEmail?: string; - /** The folder path of the email in the source mailbox. */ - path?: string; - /** An array of tags or labels associated with the email. */ - tags?: string[]; + /** A unique identifier for the email, typically assigned by the source provider. */ + id: string; + /** An optional identifier for the email thread, used to group related emails. */ + threadId?: string; + /** An array of `EmailAddress` objects representing the sender(s). */ + from: EmailAddress[]; + /** An array of `EmailAddress` objects representing the primary recipient(s). */ + to: EmailAddress[]; + /** An optional array of `EmailAddress` objects for carbon copy (CC) recipients. */ + cc?: EmailAddress[]; + /** An optional array of `EmailAddress` objects for blind carbon copy (BCC) recipients. */ + bcc?: EmailAddress[]; + /** The subject line of the email. */ + subject: string; + /** The plain text body of the email. */ + body: string; + /** The HTML version of the email body, if available. */ + html: string; + /** A map of all email headers, where keys are header names and values can be a string, an array of strings, or a complex object. */ + headers: Map; + /** An array of `EmailAttachment` objects found in the email. */ + attachments: EmailAttachment[]; + /** The date and time when the email was received. */ + receivedAt: Date; + /** An optional buffer containing the full raw EML content of the email, which is useful for archival and compliance purposes. */ + eml?: Buffer; + /** The email address of the user whose mailbox this email belongs to. */ + userEmail?: string; + /** The folder path of the email in the source mailbox. */ + path?: string; + /** An array of tags or labels associated with the email. */ + tags?: string[]; } // Define the structure of the document to be indexed in Meilisearch export interface EmailDocument { - id: string; // The unique ID of the email - from: string; - to: string[]; - cc: string[]; - bcc: string[]; - subject: string; - body: string; - attachments: { - filename: string; - content: string; // Extracted text from the attachment - }[]; - timestamp: number; - ingestionSourceId: string; - // other metadata + id: string; // The unique ID of the email + from: string; + to: string[]; + cc: string[]; + bcc: string[]; + subject: string; + body: string; + attachments: { + filename: string; + content: string; // Extracted text from the attachment + }[]; + timestamp: number; + ingestionSourceId: string; + // other metadata } diff --git a/packages/types/src/iam.types.ts b/packages/types/src/iam.types.ts index e6e26d0..58366a1 100644 --- a/packages/types/src/iam.types.ts +++ b/packages/types/src/iam.types.ts @@ -3,7 +3,7 @@ export type Action = string; export type Resource = string; export interface PolicyStatement { - Effect: 'Allow' | 'Deny'; - Action: Action[]; - Resource: Resource[]; + Effect: 'Allow' | 'Deny'; + Action: Action[]; + Resource: Resource[]; } diff --git a/packages/types/src/ingestion.types.ts b/packages/types/src/ingestion.types.ts index 715f90d..81db0fc 100644 --- a/packages/types/src/ingestion.types.ts +++ b/packages/types/src/ingestion.types.ts @@ -1,100 +1,105 @@ export type SyncState = { - google?: { - [userEmail: string]: { - historyId: string; - }; - }; - microsoft?: { - [userEmail: string]: { - deltaTokens: { [folderId: string]: string; }; - }; - }; - imap?: { - [mailboxPath: string]: { - maxUid: number; - }; - }; - lastSyncTimestamp?: string; - statusMessage?: string; + google?: { + [userEmail: string]: { + historyId: string; + }; + }; + microsoft?: { + [userEmail: string]: { + deltaTokens: { [folderId: string]: string }; + }; + }; + imap?: { + [mailboxPath: string]: { + maxUid: number; + }; + }; + lastSyncTimestamp?: string; + statusMessage?: string; }; -export type IngestionProvider = 'google_workspace' | 'microsoft_365' | 'generic_imap' | 'pst_import' | 'eml_import'; +export type IngestionProvider = + | 'google_workspace' + | 'microsoft_365' + | 'generic_imap' + | 'pst_import' + | 'eml_import'; export type IngestionStatus = - | 'active' - | 'paused' - | 'error' - | 'pending_auth' - | 'syncing' - | 'importing' - | 'auth_success' - | 'imported'; + | 'active' + | 'paused' + | 'error' + | 'pending_auth' + | 'syncing' + | 'importing' + | 'auth_success' + | 'imported'; export interface BaseIngestionCredentials { - type: IngestionProvider; + type: IngestionProvider; } export interface GenericImapCredentials extends BaseIngestionCredentials { - type: 'generic_imap'; - host: string; - port: number; - secure: boolean; - username: string; - password?: string; + type: 'generic_imap'; + host: string; + port: number; + secure: boolean; + username: string; + password?: string; } export interface GoogleWorkspaceCredentials extends BaseIngestionCredentials { - type: 'google_workspace'; - /** - * The full JSON content of the Google Service Account key. - * This should be a stringified JSON object. - */ - serviceAccountKeyJson: string; - /** - * The email of the super-admin user to impersonate for domain-wide operations. - */ - impersonatedAdminEmail: string; + type: 'google_workspace'; + /** + * The full JSON content of the Google Service Account key. + * This should be a stringified JSON object. + */ + serviceAccountKeyJson: string; + /** + * The email of the super-admin user to impersonate for domain-wide operations. + */ + impersonatedAdminEmail: string; } export interface Microsoft365Credentials extends BaseIngestionCredentials { - type: 'microsoft_365'; - clientId: string; - clientSecret: string; - tenantId: string; + type: 'microsoft_365'; + clientId: string; + clientSecret: string; + tenantId: string; } export interface PSTImportCredentials extends BaseIngestionCredentials { - type: 'pst_import'; - uploadedFileName: string; - uploadedFilePath: string; + type: 'pst_import'; + uploadedFileName: string; + uploadedFilePath: string; } export interface EMLImportCredentials extends BaseIngestionCredentials { - type: 'eml_import'; - uploadedFileName: string; - uploadedFilePath: string; + type: 'eml_import'; + uploadedFileName: string; + uploadedFilePath: string; } // Discriminated union for all possible credential types export type IngestionCredentials = - | GenericImapCredentials - | GoogleWorkspaceCredentials - | Microsoft365Credentials - | PSTImportCredentials - | EMLImportCredentials; + | GenericImapCredentials + | GoogleWorkspaceCredentials + | Microsoft365Credentials + | PSTImportCredentials + | EMLImportCredentials; export interface IngestionSource { - id: string; - name: string; - provider: IngestionProvider; - status: IngestionStatus; - createdAt: Date; - updatedAt: Date; - credentials: IngestionCredentials; - lastSyncStartedAt?: Date | null; - lastSyncFinishedAt?: Date | null; - lastSyncStatusMessage?: string | null; - syncState?: SyncState | null; + id: string; + name: string; + provider: IngestionProvider; + status: IngestionStatus; + createdAt: Date; + updatedAt: Date; + credentials: IngestionCredentials; + lastSyncStartedAt?: Date | null; + lastSyncFinishedAt?: Date | null; + lastSyncStatusMessage?: string | null; + syncState?: SyncState | null; } /** @@ -105,49 +110,48 @@ export interface IngestionSource { export type SafeIngestionSource = Omit; export interface CreateIngestionSourceDto { - name: string; - provider: IngestionProvider; - providerConfig: Record; + name: string; + provider: IngestionProvider; + providerConfig: Record; } export interface UpdateIngestionSourceDto { - name?: string; - provider?: IngestionProvider; - status?: IngestionStatus; - providerConfig?: Record; - lastSyncStartedAt?: Date; - lastSyncFinishedAt?: Date; - lastSyncStatusMessage?: string; - syncState?: SyncState; + name?: string; + provider?: IngestionProvider; + status?: IngestionStatus; + providerConfig?: Record; + lastSyncStartedAt?: Date; + lastSyncFinishedAt?: Date; + lastSyncStatusMessage?: string; + syncState?: SyncState; } export interface IContinuousSyncJob { - ingestionSourceId: string; + ingestionSourceId: string; } export interface IInitialImportJob { - ingestionSourceId: string; + ingestionSourceId: string; } export interface IProcessMailboxJob { - ingestionSourceId: string; - userEmail: string; + ingestionSourceId: string; + userEmail: string; } export interface IPstProcessingJob { - ingestionSourceId: string; - filePath: string; - originalFilename: string; + ingestionSourceId: string; + filePath: string; + originalFilename: string; } export type MailboxUser = { - id: string; - primaryEmail: string; - displayName: string; + id: string; + primaryEmail: string; + displayName: string; }; - export type ProcessMailboxError = { - error: boolean; - message: string; + error: boolean; + message: string; }; diff --git a/packages/types/src/search.types.ts b/packages/types/src/search.types.ts index 802de3e..514d531 100644 --- a/packages/types/src/search.types.ts +++ b/packages/types/src/search.types.ts @@ -3,25 +3,25 @@ import type { EmailDocument } from './email.types'; export type MatchingStrategy = 'last' | 'all' | 'frequency'; export interface SearchQuery { - query: string; - filters?: Record; - page?: number; - limit?: number; - matchingStrategy?: MatchingStrategy; + query: string; + filters?: Record; + page?: number; + limit?: number; + matchingStrategy?: MatchingStrategy; } export interface SearchHit extends EmailDocument { - _matchesPosition?: { - [key: string]: { start: number; length: number; indices?: number[]; }[]; - }; - _formatted?: Partial; + _matchesPosition?: { + [key: string]: { start: number; length: number; indices?: number[] }[]; + }; + _formatted?: Partial; } export interface SearchResult { - hits: SearchHit[]; - total: number; - page: number; - limit: number; - totalPages: number; - processingTimeMs: number; + hits: SearchHit[]; + total: number; + page: number; + limit: number; + totalPages: number; + processingTimeMs: number; } diff --git a/packages/types/src/storage.types.ts b/packages/types/src/storage.types.ts index 7f0401f..7121f68 100644 --- a/packages/types/src/storage.types.ts +++ b/packages/types/src/storage.types.ts @@ -6,66 +6,66 @@ * loading them entirely into memory. */ export interface IStorageProvider { - /** - * Stores a file at the specified path. - * @param path - The unique identifier for the file (e.g., "user-123/emails/message-abc.eml"). - * @param content - The file content as a Buffer or a ReadableStream. - * @returns A promise that resolves when the file is successfully stored. - */ - put(path: string, content: Buffer | NodeJS.ReadableStream): Promise; + /** + * Stores a file at the specified path. + * @param path - The unique identifier for the file (e.g., "user-123/emails/message-abc.eml"). + * @param content - The file content as a Buffer or a ReadableStream. + * @returns A promise that resolves when the file is successfully stored. + */ + put(path: string, content: Buffer | NodeJS.ReadableStream): Promise; - /** - * Retrieves a file from the specified path as a readable stream. - * @param path - The unique identifier for the file to retrieve. - * @returns A promise that resolves with a readable stream of the file's content. - * @throws {Error} If the file is not found. - */ - get(path: string): Promise; + /** + * Retrieves a file from the specified path as a readable stream. + * @param path - The unique identifier for the file to retrieve. + * @returns A promise that resolves with a readable stream of the file's content. + * @throws {Error} If the file is not found. + */ + get(path: string): Promise; - /** - * Deletes a file from the storage backend. - * @param path - The unique identifier for the file to delete. - * @returns A promise that resolves when the file is deleted. - */ - delete(path: string): Promise; + /** + * Deletes a file from the storage backend. + * @param path - The unique identifier for the file to delete. + * @returns A promise that resolves when the file is deleted. + */ + delete(path: string): Promise; - /** - * Checks for the existence of a file. - * @param path - The unique identifier for the file to check. - * @returns A promise that resolves with true if the file exists, false otherwise. - */ - exists(path: string): Promise; + /** + * Checks for the existence of a file. + * @param path - The unique identifier for the file to check. + * @returns A promise that resolves with true if the file exists, false otherwise. + */ + exists(path: string): Promise; } /** * Configuration for the Local Filesystem provider. */ export interface LocalStorageConfig { - type: 'local'; - // The absolute root path on the server where the archive will be stored. - rootPath: string; - openArchiverFolderName: string; + type: 'local'; + // The absolute root path on the server where the archive will be stored. + rootPath: string; + openArchiverFolderName: string; } /** * Configuration for any S3-compatible provider (AWS S3, MinIO, etc.). */ export interface S3StorageConfig { - type: 's3'; - // The API endpoint. For AWS S3, this is region-specific (e.g., 'https://s3.us-east-1.amazonaws.com'). - // For MinIO, this is the address of your MinIO server (e.g., 'http://localhost:9000'). - endpoint: string; - // The name of the bucket to use. - bucket: string; - // The access key ID for authentication. - accessKeyId: string; - // The secret access key for authentication. - secretAccessKey: string; - // The AWS region (optional but recommended for AWS S3). - region?: string; - // Force path-style addressing, required for MinIO. - forcePathStyle?: boolean; - openArchiverFolderName: string; + type: 's3'; + // The API endpoint. For AWS S3, this is region-specific (e.g., 'https://s3.us-east-1.amazonaws.com'). + // For MinIO, this is the address of your MinIO server (e.g., 'http://localhost:9000'). + endpoint: string; + // The name of the bucket to use. + bucket: string; + // The access key ID for authentication. + accessKeyId: string; + // The secret access key for authentication. + secretAccessKey: string; + // The AWS region (optional but recommended for AWS S3). + region?: string; + // Force path-style addressing, required for MinIO. + forcePathStyle?: boolean; + openArchiverFolderName: string; } export type StorageConfig = LocalStorageConfig | S3StorageConfig; diff --git a/packages/types/src/user.types.ts b/packages/types/src/user.types.ts index 7bc54ac..131417f 100644 --- a/packages/types/src/user.types.ts +++ b/packages/types/src/user.types.ts @@ -5,10 +5,10 @@ import { PolicyStatement } from './iam.types'; * This is the core user object that will be stored in the database. */ export interface User { - id: string; - first_name: string | null; - last_name: string | null; - email: string; + id: string; + first_name: string | null; + last_name: string | null; + email: string; } /** @@ -16,9 +16,9 @@ export interface User { * This is used to track a user's login status. */ export interface Session { - id: string; - userId: string; - expiresAt: Date; + id: string; + userId: string; + expiresAt: Date; } /** @@ -26,9 +26,9 @@ export interface Session { * Roles are used to group a set of permissions together. */ export interface Role { - id: string; - name: string; - policies: PolicyStatement[]; - createdAt: Date; - updatedAt: Date; + id: string; + name: string; + policies: PolicyStatement[]; + createdAt: Date; + updatedAt: Date; } diff --git a/packages/types/tsconfig.json b/packages/types/tsconfig.json index 492b402..319f4bc 100644 --- a/packages/types/tsconfig.json +++ b/packages/types/tsconfig.json @@ -1,10 +1,10 @@ { - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "outDir": "./dist", - "rootDir": "./src", - "declaration": true - }, - "include": ["src/**/*.ts"], - "exclude": ["node_modules", "dist"] + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "declaration": true + }, + "include": ["src/**/*.ts"], + "exclude": ["node_modules", "dist"] } diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 635de10..2b3476a 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -1,3 +1,3 @@ # Defines the pnpm workspace for the monorepo packages: - - 'packages/*' + - 'packages/*' diff --git a/tsconfig.base.json b/tsconfig.base.json index 0a4ae69..4a2dffd 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -1,13 +1,13 @@ { - "compilerOptions": { - "target": "ESNext", - "module": "CommonJS", - "moduleResolution": "node", - "strict": true, - "esModuleInterop": true, - "skipLibCheck": true, - "forceConsistentCasingInFileNames": true, - "resolveJsonModule": true, - "isolatedModules": true - } + "compilerOptions": { + "target": "ESNext", + "module": "CommonJS", + "moduleResolution": "node", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true, + "resolveJsonModule": true, + "isolatedModules": true + } } diff --git a/tsconfig.build.json b/tsconfig.build.json index 0dd1a3a..0be45b8 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -1,8 +1,8 @@ { - "files": [], - "references": [ - { "path": "packages/types" }, - { "path": "packages/backend" }, - { "path": "packages/frontend" } - ] + "files": [], + "references": [ + { "path": "packages/types" }, + { "path": "packages/backend" }, + { "path": "packages/frontend" } + ] }