Email thread improvement, user-defined sync frequency

2026-04-06 00:31:57 +02:00 · 2025-08-05 21:12:06 +03:00
parent f2a5b29105
commit 3201fbfe0b
9 changed files with 198 additions and 59 deletions
--- a/.env.example
+++ b/.env.example
@@ -4,6 +4,8 @@
 NODE_ENV=development
 PORT_BACKEND=4000
 PORT_FRONTEND=3000
+# The frequency of continuous email syncing. Default is every minutes, but you can change it to another value based on your needs.
+SYNC_FREQUENCY='* * * * *' 

 # --- Docker Compose Service Configuration ---
 # These variables are used by docker-compose.yml to configure the services. Leave them unchanged if you use Docker services for Postgresql, Valkey (Redis) and Meilisearch. If you decide to use your own instances of these services, you can substitute them with your own connection credentials.
@@ -20,7 +22,7 @@ MEILI_HOST=http://meilisearch:7700



-# Valkey (Redis compatible)
+# Redis (We use Valkey, which is Redis-compatible and open source)
 REDIS_HOST=valkey
 REDIS_PORT=6379
 REDIS_PASSWORD=defaultredispassword
@@ -65,3 +67,5 @@ SUPER_API_KEY=
 # IMPORTANT: Generate a secure, random 32-byte hex string for this
 # You can use `openssl rand -hex 32` to generate a key.
 ENCRYPTION_KEY=
+
+
--- a/README.md
+++ b/README.md
@@ -1,14 +1,17 @@
 # Open Archiver

-![Docker Compose](https://img.shields.io/badge/Docker%20Compose-up-4A4A4A?style=for-the-badge&logo=docker)
-![PostgreSQL](https://img.shields.io/badge/PostgreSQL-6B6B6B?style=for-the-badge&logo=postgresql)
-![Meilisearch](https://img.shields.io/badge/Meilisearch-2F2F2F?style=for-the-badge&logo=meilisearch)
+[![Docker Compose](https://img.shields.io/badge/Docker%20Compose-2496ED?style=for-the-badge&logo=docker&logoColor=white)](https://www.docker.com)
+[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-4169E1?style=for-the-badge&logo=postgresql&logoColor=white)](https://www.postgresql.org/)
+[![Meilisearch](https://img.shields.io/badge/Meilisearch-FF5A5F?style=for-the-badge&logo=meilisearch&logoColor=white)](https://www.meilisearch.com/)
+[![TypeScript](https://img.shields.io/badge/TypeScript-3178C6?style=for-the-badge&logo=typescript&logoColor=white)](https://www.typescriptlang.org/)
+[![Redis](https://img.shields.io/badge/Redis-DC382D?style=for-the-badge&logo=redis&logoColor=white)](https://redis.io)
+[![SvelteKit](https://img.shields.io/badge/SvelteKit-FF3E00?style=for-the-badge&logo=svelte&logoColor=white)](https://svelte.dev/)

-**A secure, sovereign, and affordable open-source platform for email archiving and eDiscovery.**
+**A secure, sovereign, and open-source platform for email archiving and eDiscovery.**

 Open Archiver provides a robust, self-hosted solution for archiving, storing, indexing, and searching emails from major platforms, including Google Workspace (Gmail), Microsoft 365, as well as generic IMAP-enabled email inboxes. Use Open Archiver to keep a permanent, tamper-proof record of your communication history, free from vendor lock-in.

-## Screenshots
+## 📸 Screenshots

 ![Open Archiver Preview](assets/screenshots/dashboard-1.png)
 _Dashboard_
@@ -19,7 +22,7 @@ _Archived emails_
 ![Open Archiver Preview](assets/screenshots/search.png)
 _Full-text search across all your emails and attachments_

-## Community
+## 👨‍👩‍👧‍👦 Join our community!

 We are committed to build an engaging community around Open Archiver, and we are inviting all of you to join our community on Discord to get real-time support and connect with the team.

@@ -27,7 +30,7 @@ We are committed to build an engaging community around Open Archiver, and we are

 [![Bluesky](https://img.shields.io/badge/Follow%20us%20on%20Bluesky-0265D4?style=for-the-badge&logo=bluesky&logoColor=white)](https://bsky.app/profile/openarchiver.bsky.social)

-## Live demo
+## 🚀 Live demo

 Check out the live demo here: https://demo.openarchiver.com

@@ -35,16 +38,17 @@ Username: admin@local.com

 Password: openarchiver_demo

-## Key Features
+## ✨ Key Features

 -   **Universal Ingestion**: Connect to Google Workspace, Microsoft 365, and standard IMAP servers to perform initial bulk imports and maintain continuous, real-time synchronization.
 -   **Secure & Efficient Storage**: Emails are stored in the standard `.eml` format. The system uses deduplication and compression to minimize storage costs. All data is encrypted at rest.
 -   **Pluggable Storage Backends**: Support both local filesystem storage and S3-compatible object storage (like AWS S3 or MinIO).
 -   **Powerful Search & eDiscovery**: A high-performance search engine indexes the full text of emails and attachments (PDF, DOCX, etc.).
+-   **Thread discovery**: The ability to discover if an email belongs to a thread/conversation and present the context.
 -   **Compliance & Retention**: Define granular retention policies to automatically manage the lifecycle of your data. Place legal holds on communications to prevent deletion during litigation (TBD).
 -   **Comprehensive Auditing**: An immutable audit trail logs all system activities, ensuring you have a clear record of who accessed what and when (TBD).

-## Tech Stack
+## 🛠️ Tech Stack

 Open Archiver is built on a modern, scalable, and maintainable technology stack:

@@ -55,7 +59,7 @@ Open Archiver is built on a modern, scalable, and maintainable technology stack:
 -   **Database**: PostgreSQL for metadata, user management, and audit logs
 -   **Deployment**: Docker Compose deployment

-## Deployment
+## 📦 Deployment

 ### Prerequisites

@@ -91,7 +95,7 @@ Open Archiver is built on a modern, scalable, and maintainable technology stack:
 4.  **Access the application:**
    Once the services are running, you can access the Open Archiver web interface by navigating to `http://localhost:3000` in your web browser.

-## Data Source Configuration
+## ⚙️ Data Source Configuration

 After deploying the application, you will need to configure one or more ingestion sources to begin archiving emails. Follow our detailed guides to connect to your email provider:

@@ -99,7 +103,7 @@ After deploying the application, you will need to configure one or more ingestio
 -   [Connecting to Microsoft 365](https://docs.openarchiver.com/user-guides/email-providers/imap.html)
 -   [Connecting to a Generic IMAP Server](https://docs.openarchiver.com/user-guides/email-providers/imap.html)

-## Contributing
+## 🤝 Contributing

 We welcome contributions from the community!

@@ -109,4 +113,6 @@ We welcome contributions from the community!

 Please read our `CONTRIBUTING.md` file for more details on our code of conduct and the process for submitting pull requests.

-## Star History [![Star History Chart](https://api.star-history.com/svg?repos=LogicLabs-OU/OpenArchiver&type=Date)](https://www.star-history.com/#LogicLabs-OU/OpenArchiver&Date)
+## 📈 Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=LogicLabs-OU/OpenArchiver&type=Date)](https://www.star-history.com/#LogicLabs-OU/OpenArchiver&Date)
--- a/docs/user-guides/installation.md
+++ b/docs/user-guides/installation.md
@@ -161,3 +161,45 @@ docker compose pull
 # Restart the services with the new images
 docker compose up -d
 ```
+
+## Deploying on Coolify
+
+If you are deploying Open Archiver on [Coolify](https://coolify.io/), it is recommended to let Coolify manage the Docker networks for you. This can help avoid potential routing conflicts and simplify your setup.
+
+To do this, you will need to make a small modification to your `docker-compose.yml` file.
+
+### Modify `docker-compose.yml` for Coolify
+
+1.  **Open your `docker-compose.yml` file** in a text editor.
+
+2.  **Remove all `networks` sections** from the file. This includes the network configuration for each service and the top-level network definition.
+
+    Specifically, you need to remove:
+
+    -   The `networks: - open-archiver-net` lines from the `open-archiver`, `postgres`, `valkey`, and `meilisearch` services.
+    -   The entire `networks:` block at the end of the file.
+
+    Here is an example of what to remove from a service:
+
+    ```diff
+    services:
+      open-archiver:
+        image: logiclabshq/open-archiver:latest
+        # ... other settings
+    -   networks:
+    -     - open-archiver-net
+    ```
+
+    And remove this entire block from the end of the file:
+
+    ```diff
+    - networks:
+    -   open-archiver-net:
+    -     driver: bridge
+    ```
+
+3.  **Save the modified `docker-compose.yml` file.**
+
+By removing these sections, you allow Coolify to automatically create and manage the necessary networks, ensuring that all services can communicate with each other and are correctly exposed through Coolify's reverse proxy.
+
+After making these changes, you can proceed with deploying your application on Coolify as you normally would.
--- a/packages/backend/src/config/app.ts
+++ b/packages/backend/src/config/app.ts
@@ -5,4 +5,5 @@ export const app = {
    port: process.env.PORT_BACKEND ? parseInt(process.env.PORT_BACKEND, 10) : 4000,
    encryptionKey: process.env.ENCRYPTION_KEY,
    isDemo: process.env.IS_DEMO === 'true',
+    syncFrequency: process.env.SYNC_FREQUENCY || '* * * * *' //default to 1 minute
 };
--- a/packages/backend/src/jobs/schedulers/sync-scheduler.ts
+++ b/packages/backend/src/jobs/schedulers/sync-scheduler.ts
@@ -1,5 +1,7 @@
 import { ingestionQueue } from '../queues';

+import { config } from '../../config';
+
 const scheduleContinuousSync = async () => {
    // This job will run every 15 minutes
    await ingestionQueue.add(
@@ -7,7 +9,7 @@ const scheduleContinuousSync = async () => {
        {},
        {
            repeat: {
-                pattern: '* * * * *', // Every 1 minute
+                pattern: config.app.syncFrequency
            },
        }
    );
--- a/packages/frontend/src/lib/components/custom/EmailThread.svelte
+++ b/packages/frontend/src/lib/components/custom/EmailThread.svelte
@@ -1,6 +1,7 @@
 <script lang="ts">
 	import { goto } from '$app/navigation';
 	import type { ArchivedEmail } from '@open-archiver/types';
+	import { ScrollArea } from '$lib/components/ui/scroll-area/index.js';

 	let {
 		thread,
@@ -12,51 +13,53 @@
 </script>

 <div>
-	<div class="relative border-l-2 border-gray-200 pl-6">
-		{#if thread}
-			{#each thread as item, i (item.id)}
-				<div class="mb-8">
-					<span
-						class="absolute -left-3 flex h-6 w-6 items-center justify-center rounded-full bg-gray-200 ring-8 ring-white"
-					>
-						<svg
-							class="h-3 w-3 text-gray-600"
-							fill="currentColor"
-							viewBox="0 0 20 20"
-							xmlns="http://www.w3.org/2000/svg"
-							><path
-								fill-rule="evenodd"
-								d="M6 2a1 1 0 00-1 1v1H4a2 2 0 00-2 2v10a2 2 0 002 2h12a2 2 0 002-2V6a2 2 0 00-2-2h-1V3a1 1 0 10-2 0v1H7V3a1 1 0 00-1-1zm0 5a1 1 0 000 2h8a1 1 0 100-2H6z"
-								clip-rule="evenodd"
-							></path></svg
+	<ScrollArea class="max-h-120 -ml-3 overflow-y-scroll">
+		<div class="relative ml-3 border-l-2 border-gray-200 pl-6">
+			{#if thread}
+				{#each thread as item, i (item.id)}
+					<div class="mb-8">
+						<span
+							class="absolute -left-3 flex h-6 w-6 items-center justify-center rounded-full bg-gray-200 ring-8 ring-white"
 						>
-					</span>
-					<h4
-						class:font-bold={item.id === currentEmailId}
-						class="text-md mb-2 {item.id !== currentEmailId
-							? 'text-blue-500 hover:underline'
-							: 'text-gray-900'}"
-					>
-						{#if item.id !== currentEmailId}
-							<a
-								href="/dashboard/archived-emails/{item.id}"
-								onclick={(e) => {
-									e.preventDefault();
-									goto(`/dashboard/archived-emails/${item.id}`, {
-										invalidateAll: true
-									});
-								}}>{item.subject || 'No Subject'}</a
+							<svg
+								class="h-3 w-3 text-gray-600"
+								fill="currentColor"
+								viewBox="0 0 20 20"
+								xmlns="http://www.w3.org/2000/svg"
+								><path
+									fill-rule="evenodd"
+									d="M6 2a1 1 0 00-1 1v1H4a2 2 0 00-2 2v10a2 2 0 002 2h12a2 2 0 002-2V6a2 2 0 00-2-2h-1V3a1 1 0 10-2 0v1H7V3a1 1 0 00-1-1zm0 5a1 1 0 000 2h8a1 1 0 100-2H6z"
+									clip-rule="evenodd"
+								></path></svg
 							>
-						{:else}
-							{item.subject || 'No Subject'}
-						{/if}
-					</h4>
-					<div class="flex flex-col space-y-2 text-sm font-normal leading-none text-gray-400">
-						<span>From: {item.senderEmail}</span>
-						<time class="">{new Date(item.sentAt).toLocaleString()}</time>
+						</span>
+						<h4
+							class:font-bold={item.id === currentEmailId}
+							class="text-md mb-2 {item.id !== currentEmailId
+								? 'text-blue-500 hover:underline'
+								: 'text-gray-900'}"
+						>
+							{#if item.id !== currentEmailId}
+								<a
+									href="/dashboard/archived-emails/{item.id}"
+									onclick={(e) => {
+										e.preventDefault();
+										goto(`/dashboard/archived-emails/${item.id}`, {
+											invalidateAll: true
+										});
+									}}>{item.subject || 'No Subject'}</a
+								>
+							{:else}
+								{item.subject || 'No Subject'}
+							{/if}
+						</h4>
+						<div class="flex flex-col space-y-2 text-sm font-normal leading-none text-gray-400">
+							<span>From: {item.senderEmail}</span>
+							<time class="">{new Date(item.sentAt).toLocaleString()}</time>
+						</div>
 					</div>
-				</div>
-			{/each}
-		{/if}
-	</div>
+				{/each}
+			{/if}
+		</div>
+	</ScrollArea>
 </div>
--- a/packages/frontend/src/lib/components/ui/scroll-area/index.ts
+++ b/packages/frontend/src/lib/components/ui/scroll-area/index.ts
@@ -0,0 +1,10 @@
+import Scrollbar from "./scroll-area-scrollbar.svelte";
+import Root from "./scroll-area.svelte";
+
+export {
+	Root,
+	Scrollbar,
+	//,
+	Root as ScrollArea,
+	Scrollbar as ScrollAreaScrollbar,
+};
--- a/packages/frontend/src/lib/components/ui/scroll-area/scroll-area-scrollbar.svelte
+++ b/packages/frontend/src/lib/components/ui/scroll-area/scroll-area-scrollbar.svelte
@@ -0,0 +1,31 @@
+<script lang="ts">
+	import { ScrollArea as ScrollAreaPrimitive } from "bits-ui";
+	import { cn, type WithoutChild } from "$lib/utils.js";
+
+	let {
+		ref = $bindable(null),
+		class: className,
+		orientation = "vertical",
+		children,
+		...restProps
+	}: WithoutChild<ScrollAreaPrimitive.ScrollbarProps> = $props();
+</script>
+
+<ScrollAreaPrimitive.Scrollbar
+	bind:ref
+	data-slot="scroll-area-scrollbar"
+	{orientation}
+	class={cn(
+		"flex touch-none select-none p-px transition-colors",
+		orientation === "vertical" && "h-full w-2.5 border-l border-l-transparent",
+		orientation === "horizontal" && "h-2.5 flex-col border-t border-t-transparent",
+		className
+	)}
+	{...restProps}
+>
+	{@render children?.()}
+	<ScrollAreaPrimitive.Thumb
+		data-slot="scroll-area-thumb"
+		class="bg-border relative flex-1 rounded-full"
+	/>
+</ScrollAreaPrimitive.Scrollbar>
--- a/packages/frontend/src/lib/components/ui/scroll-area/scroll-area.svelte
+++ b/packages/frontend/src/lib/components/ui/scroll-area/scroll-area.svelte
@@ -0,0 +1,40 @@
+<script lang="ts">
+	import { ScrollArea as ScrollAreaPrimitive } from "bits-ui";
+	import { Scrollbar } from "./index.js";
+	import { cn, type WithoutChild } from "$lib/utils.js";
+
+	let {
+		ref = $bindable(null),
+		class: className,
+		orientation = "vertical",
+		scrollbarXClasses = "",
+		scrollbarYClasses = "",
+		children,
+		...restProps
+	}: WithoutChild<ScrollAreaPrimitive.RootProps> & {
+		orientation?: "vertical" | "horizontal" | "both" | undefined;
+		scrollbarXClasses?: string | undefined;
+		scrollbarYClasses?: string | undefined;
+	} = $props();
+</script>
+
+<ScrollAreaPrimitive.Root
+	bind:ref
+	data-slot="scroll-area"
+	class={cn("relative", className)}
+	{...restProps}
+>
+	<ScrollAreaPrimitive.Viewport
+		data-slot="scroll-area-viewport"
+		class="ring-ring/10 dark:ring-ring/20 dark:outline-ring/40 outline-ring/50 size-full rounded-[inherit] transition-[color,box-shadow] focus-visible:outline-1 focus-visible:ring-4"
+	>
+		{@render children?.()}
+	</ScrollAreaPrimitive.Viewport>
+	{#if orientation === "vertical" || orientation === "both"}
+		<Scrollbar orientation="vertical" class={scrollbarYClasses} />
+	{/if}
+	{#if orientation === "horizontal" || orientation === "both"}
+		<Scrollbar orientation="horizontal" class={scrollbarXClasses} />
+	{/if}
+	<ScrollAreaPrimitive.Corner />
+</ScrollAreaPrimitive.Root>