Compare commits

...

8 Commits
docs ... v0.3.3

Author SHA1 Message Date
Wei S.
e9a65f9672 feat: Add Mbox ingestion (#117)
This commit introduces two major features:

1.  **Mbox File Ingestion:**
    Users can now ingest emails from Mbox files (`.mbox`). A new Mbox connector has been implemented on the backend, and the user interface has been updated to support creating Mbox ingestion sources. Documentation for this new provider has also been added.

Additionally, this commit includes new documentation for upgrading and migrating Open Archiver.

Co-authored-by: Wayne <5291640+ringoinca@users.noreply.github.com>
2025-09-16 20:30:22 +03:00
Wei S.
ce3f379b7a Update issue templates (#110) 2025-09-14 16:25:13 +03:00
Wei S.
37a778cb6d chore(deps): Update dependencies across packages (#105)
This commit updates several dependencies in the frontend and backend packages.

- **Backend:**
  - Upgrades `xlsx` to version `0.20.3` by pointing to the official CDN URL. This ensures usage of the community edition with a permissive license.
  - Removes the unused `bull-board` development dependency.

- **Frontend:**
  - Upgrades `@sveltejs/kit` from `^2.16.0` to `^2.38.1` to stay current with the latest features and fixes.

Co-authored-by: Wayne <5291640+ringoinca@users.noreply.github.com>
2025-09-11 22:07:35 +03:00
Wei S.
26a760b232 Create FUNDING.yml (#102) 2025-09-10 17:09:13 +03:00
Wei S.
6be0774bc4 Display versions: Add new version notification in footer (#101)
* feat: Add new version notification in footer

This commit implements a system to check for new application versions and notify the user.

On page load, the server-side code now fetches the latest release from the GitHub repository API. It uses `semver` to compare the current application version with the latest release tag.

If a newer version is available, an alert is displayed in the footer with a link to the release page. The current application version is also now displayed in the footer. The version check is cached for one hour to minimize API requests.

* Modify version notification

* current version 0.3.1

* Resolve conflicts

* Code formatting

---------

Co-authored-by: Wayne <5291640+ringoinca@users.noreply.github.com>
2025-09-10 12:09:12 +03:00
Wei S.
4a23f8f29f feat: Add new version notification in footer (#99)
This commit implements a system to check for new application versions and notify the user.

On page load, the server-side code now fetches the latest release from the GitHub repository API. It uses `semver` to compare the current application version with the latest release tag.

If a newer version is available, an alert is displayed in the footer with a link to the release page. The current application version is also now displayed in the footer. The version check is cached for one hour to minimize API requests.

Co-authored-by: Wayne <5291640+ringoinca@users.noreply.github.com>
2025-09-09 23:36:35 +03:00
albanobattistella
074256ed59 Update it.json (#90) 2025-09-07 23:44:08 +03:00
Wei S.
7d178d786b Docs: code formatting (#92)
* Adding rate limiting docs

* update rate limiting docs

* Resolve conflict

* Code formatting

---------

Co-authored-by: Wayne <5291640+ringoinca@users.noreply.github.com>
2025-09-06 18:06:59 +03:00
46 changed files with 4630 additions and 3238 deletions

1
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1 @@
github: [wayneshn]

32
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,32 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
5. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**System:**
- Open Archiver Version:
**Relevant logs:**
Any relevant logs (Redact sensitive information)
**Additional context**
Add any other context about the problem here.

View File

@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is.
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

View File

@@ -46,12 +46,14 @@ Password: openarchiver_demo
- Microsoft 365
- PST files
- Zipped .eml files
- Mbox files
- **Secure & Efficient Storage**: Emails are stored in the standard `.eml` format. The system uses deduplication and compression to minimize storage costs. All data is encrypted at rest.
- **Pluggable Storage Backends**: Support both local filesystem storage and S3-compatible object storage (like AWS S3 or MinIO).
- **Powerful Search & eDiscovery**: A high-performance search engine indexes the full text of emails and attachments (PDF, DOCX, etc.).
- **Thread discovery**: The ability to discover if an email belongs to a thread/conversation and present the context.
- **Compliance & Retention**: Define granular retention policies to automatically manage the lifecycle of your data. Place legal holds on communications to prevent deletion during litigation (TBD).
- **File Hash and Encryption**: Email and attachment file hash values are stored in the meta database upon ingestion, meaning any attempt to alter the file content will be identified, ensuring legal and regulatory compliance.
- **Comprehensive Auditing**: An immutable audit trail logs all system activities, ensuring you have a clear record of who accessed what and when (TBD).
## 🛠️ Tech Stack

View File

@@ -52,6 +52,7 @@ export default defineConfig({
},
{ text: 'EML Import', link: '/user-guides/email-providers/eml' },
{ text: 'PST Import', link: '/user-guides/email-providers/pst' },
{ text: 'Mbox Import', link: '/user-guides/email-providers/mbox' },
],
},
{
@@ -64,6 +65,20 @@ export default defineConfig({
},
],
},
{
text: 'Upgrading and Migration',
collapsed: true,
items: [
{
text: 'Upgrading',
link: '/user-guides/upgrade-and-migration/upgrade',
},
{
text: 'Meilisearch Upgrade',
link: '/user-guides/upgrade-and-migration/meilisearch-upgrade',
},
],
},
],
},
{

View File

@@ -9,3 +9,4 @@ Choose your provider from the list below to get started:
- [Generic IMAP Server](./imap.md)
- [EML Import](./eml.md)
- [PST Import](./pst.md)
- [Mbox Import](./mbox.md)

View File

@@ -0,0 +1,29 @@
# Mbox Ingestion
Mbox is a common format for storing email messages. This guide will walk you through the process of ingesting mbox files into OpenArchiver.
## 1. Exporting from Your Email Client
Most email clients that support mbox exports will allow you to export a folder of emails as a single `.mbox` file. Here are the general steps:
- **Mozilla Thunderbird**: Right-click on a folder, select **ImportExportTools NG**, and then choose **Export folder**.
- **Gmail**: You can use Google Takeout to export your emails in mbox format.
- **Other Clients**: Refer to your email client's documentation for instructions on how to export emails to an mbox file.
## 2. Uploading to OpenArchiver
Once you have your `.mbox` file, you can upload it to OpenArchiver through the web interface.
1. Navigate to the **Ingestion** page.
2. Click on the **New Ingestion** button.
3. Select **Mbox** as the source type.
4. Upload your `.mbox` file.
## 3. Folder Structure
OpenArchiver will attempt to preserve the original folder structure of your emails. This is done by inspecting the following email headers:
- `X-Gmail-Labels`: Used by Gmail to store labels.
- `X-Folder`: A custom header used by some email clients like Thunderbird.
If neither of these headers is present, the emails will be ingested into the root of the archive.

View File

@@ -138,7 +138,9 @@ docker compose ps
Once the services are running, you can access the Open Archiver web interface by navigating to `http://localhost:3000` in your web browser.
You can log in with the `ADMIN_EMAIL` and `ADMIN_PASSWORD` you configured in your `.env` file.
Upon first visit, you will be redirected to the `/setup` page where you can set up your admin account. Make sure you are the first person who accesses the instance.
If you are not redirected to the `/setup` page but instead see the login page, there might be something wrong with the database. Restart the service and try again.
## 5. Next Steps
@@ -212,9 +214,9 @@ If you are using local storage to store your emails, based on your `docker-compo
Run this command to see all the volumes on your system:
```bash
docker volume ls
```
```bash
docker volume ls
```
2. **Identify the correct volume**:
@@ -224,28 +226,28 @@ Look through the list for a volume name that ends with `_archiver-data`. The par
Once you've identified the correct volume name, use it in the `inspect` command. For example:
```bash
docker volume inspect <your_volume_name_here>
```
```bash
docker volume inspect <your_volume_name_here>
```
This will give you the correct `Mountpoint` path where your data is being stored. It will look something like this (the exact path will vary depending on your system):
```json
{
"CreatedAt": "2025-07-25T11:22:19Z",
"Driver": "local",
"Labels": {
"com.docker.compose.config-hash": "---",
"com.docker.compose.project": "---",
"com.docker.compose.version": "2.38.2",
"com.docker.compose.volume": "us8wwos0o4ok4go4gc8cog84_archiver-data"
},
"Mountpoint": "/var/lib/docker/volumes/us8wwos0o4ok4go4gc8cog84_archiver-data/_data",
"Name": "us8wwos0o4ok4go4gc8cog84_archiver-data",
"Options": null,
"Scope": "local"
}
```
```json
{
"CreatedAt": "2025-07-25T11:22:19Z",
"Driver": "local",
"Labels": {
"com.docker.compose.config-hash": "---",
"com.docker.compose.project": "---",
"com.docker.compose.version": "2.38.2",
"com.docker.compose.volume": "us8wwos0o4ok4go4gc8cog84_archiver-data"
},
"Mountpoint": "/var/lib/docker/volumes/us8wwos0o4ok4go4gc8cog84_archiver-data/_data",
"Name": "us8wwos0o4ok4go4gc8cog84_archiver-data",
"Options": null,
"Scope": "local"
}
```
In this example, the data is located at `/var/lib/docker/volumes/us8wwos0o4ok4go4gc8cog84_archiver-data/_data`. You can then `cd` into that directory to see your files.
@@ -259,44 +261,44 @@ Heres how you can do it:
Open the `docker-compose.yml` file and find the `open-archiver` service. You're going to change the `volumes` section.
**Change this:**
**Change this:**
```yaml
services:
open-archiver:
# ... other config
volumes:
- archiver-data:/var/data/open-archiver
```
```yaml
services:
open-archiver:
# ... other config
volumes:
- archiver-data:/var/data/open-archiver
```
**To this:**
**To this:**
```yaml
services:
open-archiver:
# ... other config
volumes:
- ./data/open-archiver:/var/data/open-archiver
```
```yaml
services:
open-archiver:
# ... other config
volumes:
- ./data/open-archiver:/var/data/open-archiver
```
You'll also want to remove the `archiver-data` volume definition at the bottom of the file, since it's no longer needed.
**Remove this whole block:**
**Remove this whole block:**
```yaml
volumes:
# ... other volumes
archiver-data:
driver: local
```
```yaml
volumes:
# ... other volumes
archiver-data:
driver: local
```
2. **Restart your containers**:
After you've saved the changes, run the following command in your terminal to apply them. The `--force-recreate` flag will ensure the container is recreated with the new volume settings.
```bash
docker-compose up -d --force-recreate
```
```bash
docker-compose up -d --force-recreate
```
After this, any new data will be saved directly into the `./data/open-archiver` folder in your project directory.

View File

@@ -0,0 +1,93 @@
# Upgrading Meilisearch
Meilisearch, the search engine used by Open Archiver, requires a manual data migration process when upgrading to a new version. This is because Meilisearch databases are only compatible with the specific version that created them.
If an Open Archiver upgrade includes a major Meilisearch version change, you will need to migrate your search index by following the process below.
## Migration Process Overview
For self-hosted instances using Docker Compose (as recommended), the migration process involves creating a data dump from your current Meilisearch instance, upgrading the Docker image, and then importing that dump into the new version.
### Step 1: Create a Dump
Before upgrading, you must create a dump of your existing Meilisearch data. You can do this by sending a POST request to the `/dumps` endpoint of the Meilisearch API.
1. **Find your Meilisearch container name**:
```bash
docker compose ps
```
Look for the service name that corresponds to Meilisearch, usually `meilisearch`.
2. **Execute the dump command**:
You will need your Meilisearch Admin API key, which can be found in your `.env` file as `MEILI_MASTER_KEY`.
```bash
curl -X POST 'http://localhost:7700/dumps' \
-H "Authorization: Bearer YOUR_MEILI_MASTER_KEY"
```
This will start the dump creation process. The dump file will be created inside the `meili_data` volume used by the Meilisearch container.
3. **Monitor the dump status**:
The dump creation request returns a `taskUid`. You can use this to check the status of the dump.
For more details on dump and import, see the [official Meilisearch documentation](https://www.meilisearch.com/docs/learn/update_and_migration/updating).
### Step 2: Upgrade Your Open Archiver Instance
Once the dump is successfully created, you can proceed with the standard Open Archiver upgrade process.
1. **Pull the latest changes and Docker images**:
```bash
git pull
docker compose pull
```
2. **Stop the running services**:
```bash
docker compose down
```
### Step 3: Import the Dump
Now, you need to restart the services while telling Meilisearch to import from your dump file.
1. **Modify `docker-compose.yml`**:
You need to temporarily add the `--import-dump` flag to the Meilisearch service command. Find the `meilisearch` service in your `docker-compose.yml` and modify the `command` section.
You will need the name of your dump file. It will be a `.dump` file located in the directory mapped to `/meili_data` inside the container.
```yaml
services:
meilisearch:
# ... other service config
command:
[
'--master-key=${MEILI_MASTER_KEY}',
'--env=production',
'--import-dump=/meili_data/dumps/YOUR_DUMP_FILE.dump',
]
```
2. **Restart the services**:
```bash
docker compose up -d
```
Meilisearch will now start and import the data from the dump file. This may take some time depending on the size of your index.
### Step 4: Clean Up
Once the import is complete and you have verified that your search is working correctly, you should remove the `--import-dump` flag from your `docker-compose.yml` to prevent it from running on every startup.
1. **Remove the `--import-dump` line** from the `command` section of the `meilisearch` service in `docker-compose.yml`.
2. **Restart the services** one last time:
```bash
docker compose up -d
```
Your Meilisearch instance is now upgraded and running with your migrated data.
For more advanced scenarios or troubleshooting, please refer to the **[official Meilisearch migration guide](https://www.meilisearch.com/docs/learn/update_and_migration/updating)**.

View File

@@ -0,0 +1,42 @@
# Upgrading Your Instance
This guide provides instructions for upgrading your Open Archiver instance to the latest version.
## Checking for New Versions
Open Archiver automatically checks for new versions and will display a notification in the footer of the web interface when an update is available. You can find a list of all releases and their release notes on the [GitHub Releases](https://github.com/LogicLabs-OU/OpenArchiver/releases) page.
## Upgrading Your Instance
To upgrade your Open Archiver instance, follow these steps:
1. **Pull the latest changes from the repository**:
```bash
git pull
```
2. **Pull the latest Docker images**:
```bash
docker compose pull
```
3. **Restart the services with the new images**:
```bash
docker compose up -d
```
This will restart your Open Archiver instance with the latest version of the application.
## Migrating Data
When you upgrade to a new version, database migrations are applied automatically when the application starts up. This ensures that your database schema is always up-to-date with the latest version of the application.
No manual intervention is required for database migrations.
## Upgrading Meilisearch
When an Open Archiver update includes a major version change for Meilisearch, you will need to manually migrate your search data. This process is not covered by the standard upgrade commands.
For detailed instructions, please see the [Meilisearch Upgrade Guide](./meilisearch-upgrade.md).

77
open-archiver.yml Normal file
View File

@@ -0,0 +1,77 @@
# documentation: https://openarchiver.com
# slogan: A self-hosted, open-source email archiving solution with full-text search capability.
# tags: email archiving,email,compliance,search
# logo: svgs/openarchiver.svg
# port: 3000
services:
open-archiver:
image: logiclabshq/open-archiver:latest
environment:
- SERVICE_URL_3000
- SERVICE_URL=${SERVICE_URL_3000}
- PORT_BACKEND=${PORT_BACKEND:-4000}
- PORT_FRONTEND=${PORT_FRONTEND:-3000}
- NODE_ENV=${NODE_ENV:-production}
- SYNC_FREQUENCY=${SYNC_FREQUENCY:-* * * * *}
- POSTGRES_DB=${POSTGRES_DB:-open_archive}
- POSTGRES_USER=${POSTGRES_USER:-admin}
- POSTGRES_PASSWORD=${SERVICE_PASSWORD_POSTGRES}
- DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}
- MEILI_MASTER_KEY=${SERVICE_PASSWORD_MEILISEARCH}
- MEILI_HOST=http://meilisearch:7700
- REDIS_HOST=valkey
- REDIS_PORT=6379
- REDIS_PASSWORD=${SERVICE_PASSWORD_VALKEY}
- REDIS_TLS_ENABLED=false
- STORAGE_TYPE=${STORAGE_TYPE:-local}
- STORAGE_LOCAL_ROOT_PATH=${STORAGE_LOCAL_ROOT_PATH:-/var/data/open-archiver}
- BODY_SIZE_LIMIT=${BODY_SIZE_LIMIT:-100M}
- STORAGE_S3_ENDPOINT=${STORAGE_S3_ENDPOINT}
- STORAGE_S3_BUCKET=${STORAGE_S3_BUCKET}
- STORAGE_S3_ACCESS_KEY_ID=${STORAGE_S3_ACCESS_KEY_ID}
- STORAGE_S3_SECRET_ACCESS_KEY=${STORAGE_S3_SECRET_ACCESS_KEY}
- STORAGE_S3_REGION=${STORAGE_S3_REGION}
- STORAGE_S3_FORCE_PATH_STYLE=${STORAGE_S3_FORCE_PATH_STYLE:-false}
- JWT_SECRET=${SERVICE_BASE64_128_JWT}
- JWT_EXPIRES_IN=${JWT_EXPIRES_IN:-7d}
- ENCRYPTION_KEY=${SERVICE_BASE64_64_ENCRYPTIONKEY}
- RATE_LIMIT_WINDOW_MS=${RATE_LIMIT_WINDOW_MS:-60000}
- RATE_LIMIT_MAX_REQUESTS=${RATE_LIMIT_MAX_REQUESTS:-100}
volumes:
- archiver-data:/var/data/open-archiver
depends_on:
postgres:
condition: service_healthy
valkey:
condition: service_started
meilisearch:
condition: service_started
postgres:
image: postgres:17-alpine
environment:
- POSTGRES_DB=${POSTGRES_DB}
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${SERVICE_PASSWORD_POSTGRES}
- LC_ALL=C
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ['CMD-SHELL', 'pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}']
interval: 10s
timeout: 20s
retries: 10
valkey:
image: valkey/valkey:8-alpine
command: valkey-server --requirepass ${SERVICE_PASSWORD_VALKEY}
volumes:
- valkeydata:/data
meilisearch:
image: getmeili/meilisearch:v1.15
environment:
- MEILI_MASTER_KEY=${SERVICE_PASSWORD_MEILISEARCH}
volumes:
- meilidata:/meili_data

View File

@@ -1,5 +1,6 @@
{
"name": "open-archiver",
"version": "0.3.3",
"private": true,
"scripts": {
"dev": "dotenv -- pnpm --filter \"./packages/*\" --parallel dev",

View File

@@ -59,7 +59,7 @@
"reflect-metadata": "^0.2.2",
"sqlite3": "^5.1.7",
"tsconfig-paths": "^4.2.0",
"xlsx": "^0.18.5",
"xlsx": "https://cdn.sheetjs.com/xlsx-0.20.3/xlsx-0.20.3.tgz",
"yauzl": "^3.2.0",
"zod": "^4.1.5"
},
@@ -74,7 +74,6 @@
"@types/multer": "^2.0.0",
"@types/node": "^24.0.12",
"@types/yauzl": "^2.10.3",
"bull-board": "^2.1.3",
"ts-node-dev": "^2.0.0",
"typescript": "^5.8.3"
}

View File

@@ -4,54 +4,63 @@ import { z } from 'zod';
import { config } from '../../config';
const generateApiKeySchema = z.object({
name: z.string().min(1, 'API kay name must be more than 1 characters').max(255, 'API kay name must not be more than 255 characters'),
expiresInDays: z.number().int().positive('Only positive number is allowed').max(730, "The API key must expire within 2 years / 730 days."),
name: z
.string()
.min(1, 'API kay name must be more than 1 characters')
.max(255, 'API kay name must not be more than 255 characters'),
expiresInDays: z
.number()
.int()
.positive('Only positive number is allowed')
.max(730, 'The API key must expire within 2 years / 730 days.'),
});
export class ApiKeyController {
public async generateApiKey(req: Request, res: Response) {
if (config.app.isDemo) {
return res.status(403).json({ message: req.t('errors.demoMode') });
}
try {
const { name, expiresInDays } = generateApiKeySchema.parse(req.body);
if (!req.user || !req.user.sub) {
return res.status(401).json({ message: 'Unauthorized' });
}
const userId = req.user.sub;
public async generateApiKey(req: Request, res: Response) {
if (config.app.isDemo) {
return res.status(403).json({ message: req.t('errors.demoMode') });
}
try {
const { name, expiresInDays } = generateApiKeySchema.parse(req.body);
if (!req.user || !req.user.sub) {
return res.status(401).json({ message: 'Unauthorized' });
}
const userId = req.user.sub;
const key = await ApiKeyService.generate(userId, name, expiresInDays);
const key = await ApiKeyService.generate(userId, name, expiresInDays);
res.status(201).json({ key });
} catch (error) {
if (error instanceof z.ZodError) {
return res.status(400).json({ message: req.t('api.requestBodyInvalid'), errors: error.message });
}
res.status(500).json({ message: req.t('errors.internalServerError') });
}
}
res.status(201).json({ key });
} catch (error) {
if (error instanceof z.ZodError) {
return res
.status(400)
.json({ message: req.t('api.requestBodyInvalid'), errors: error.message });
}
res.status(500).json({ message: req.t('errors.internalServerError') });
}
}
public async getApiKeys(req: Request, res: Response) {
if (!req.user || !req.user.sub) {
return res.status(401).json({ message: 'Unauthorized' });
}
const userId = req.user.sub;
const keys = await ApiKeyService.getKeys(userId);
public async getApiKeys(req: Request, res: Response) {
if (!req.user || !req.user.sub) {
return res.status(401).json({ message: 'Unauthorized' });
}
const userId = req.user.sub;
const keys = await ApiKeyService.getKeys(userId);
res.status(200).json(keys);
}
res.status(200).json(keys);
}
public async deleteApiKey(req: Request, res: Response) {
if (config.app.isDemo) {
return res.status(403).json({ message: req.t('errors.demoMode') });
}
const { id } = req.params;
if (!req.user || !req.user.sub) {
return res.status(401).json({ message: 'Unauthorized' });
}
const userId = req.user.sub;
await ApiKeyService.deleteKey(id, userId);
public async deleteApiKey(req: Request, res: Response) {
if (config.app.isDemo) {
return res.status(403).json({ message: req.t('errors.demoMode') });
}
const { id } = req.params;
if (!req.user || !req.user.sub) {
return res.status(401).json({ message: 'Unauthorized' });
}
const userId = req.user.sub;
await ApiKeyService.deleteKey(id, userId);
res.status(204).send({ message: req.t('apiKeys.deleteSuccess') });
}
res.status(204).send({ message: req.t('apiKeys.deleteSuccess') });
}
}

View File

@@ -7,6 +7,7 @@ import { config } from '../../config/index';
export const uploadFile = async (req: Request, res: Response) => {
const storage = new StorageService();
const bb = busboy({ headers: req.headers });
const uploads: Promise<void>[] = [];
let filePath = '';
let originalFilename = '';
@@ -14,10 +15,11 @@ export const uploadFile = async (req: Request, res: Response) => {
originalFilename = filename.filename;
const uuid = randomUUID();
filePath = `${config.storage.openArchiverFolderName}/tmp/${uuid}-${originalFilename}`;
storage.put(filePath, file);
uploads.push(storage.put(filePath, file));
});
bb.on('finish', () => {
bb.on('finish', async () => {
await Promise.all(uploads);
res.json({ filePath });
});

View File

@@ -8,9 +8,9 @@ export const rateLimiter = rateLimit({
max: config.api.rateLimit.max,
message: {
status: 429,
message: `Too many requests from this IP, please try again after ${windowInMinutes} minutes`
message: `Too many requests from this IP, please try again after ${windowInMinutes} minutes`,
},
statusCode: 429,
standardHeaders: true,
legacyHeaders: false
legacyHeaders: false,
});

View File

@@ -32,7 +32,7 @@ export const requireAuth = (authService: AuthService) => {
req.user = {
sub: user.id,
email: user.email,
roles: user.role ? [user.role.name] : []
roles: user.role ? [user.role.name] : [],
};
return next();
}

View File

@@ -4,12 +4,12 @@ import { requireAuth } from '../middleware/requireAuth';
import { AuthService } from '../../services/AuthService';
export const apiKeyRoutes = (authService: AuthService) => {
const router = Router();
const controller = new ApiKeyController();
const router = Router();
const controller = new ApiKeyController();
router.post('/', requireAuth(authService), controller.generateApiKey);
router.get('/', requireAuth(authService), controller.getApiKeys);
router.delete('/:id', requireAuth(authService), controller.deleteApiKey);
router.post('/', requireAuth(authService), controller.generateApiKey);
router.get('/', requireAuth(authService), controller.getApiKeys);
router.delete('/:id', requireAuth(authService), controller.deleteApiKey);
return router;
return router;
};

View File

@@ -1,8 +1,12 @@
import 'dotenv/config';
export const apiConfig = {
rateLimit: {
windowMs: process.env.RATE_LIMIT_WINDOW_MS ? parseInt(process.env.RATE_LIMIT_WINDOW_MS, 10) : 1 * 60 * 1000, // 1 minutes
max: process.env.RATE_LIMIT_MAX_REQUESTS ? parseInt(process.env.RATE_LIMIT_MAX_REQUESTS, 10) : 100, // limit each IP to 100 requests per windowMs
}
rateLimit: {
windowMs: process.env.RATE_LIMIT_WINDOW_MS
? parseInt(process.env.RATE_LIMIT_WINDOW_MS, 10)
: 1 * 60 * 1000, // 1 minutes
max: process.env.RATE_LIMIT_MAX_REQUESTS
? parseInt(process.env.RATE_LIMIT_MAX_REQUESTS, 10)
: 100, // limit each IP to 100 requests per windowMs
},
};

View File

@@ -2,6 +2,7 @@ import pino from 'pino';
export const logger = pino({
level: process.env.LOG_LEVEL || 'info',
redact: ['password'],
transport: {
target: 'pino-pretty',
options: {

View File

@@ -0,0 +1 @@
ALTER TYPE "public"."ingestion_provider" ADD VALUE 'mbox_import';

File diff suppressed because it is too large Load Diff

View File

@@ -141,6 +141,13 @@
"when": 1756937533843,
"tag": "0019_confused_scream",
"breakpoints": true
},
{
"idx": 20,
"version": "7",
"when": 1757860242528,
"tag": "0020_panoramic_wolverine",
"breakpoints": true
}
]
}

View File

@@ -2,14 +2,14 @@ import { pgTable, text, timestamp, uuid } from 'drizzle-orm/pg-core';
import { users } from './users';
export const apiKeys = pgTable('api_keys', {
id: uuid('id').primaryKey().defaultRandom(),
name: text('name').notNull(),
userId: uuid('user_id')
.notNull()
.references(() => users.id, { onDelete: 'cascade' }),
key: text('key').notNull(), // Encrypted API key
keyHash: text('key_hash').notNull(),
expiresAt: timestamp('expires_at', { withTimezone: true, mode: 'date' }).notNull(),
createdAt: timestamp('created_at').defaultNow().notNull(),
updatedAt: timestamp('updated_at').defaultNow().notNull(),
id: uuid('id').primaryKey().defaultRandom(),
name: text('name').notNull(),
userId: uuid('user_id')
.notNull()
.references(() => users.id, { onDelete: 'cascade' }),
key: text('key').notNull(), // Encrypted API key
keyHash: text('key_hash').notNull(),
expiresAt: timestamp('expires_at', { withTimezone: true, mode: 'date' }).notNull(),
createdAt: timestamp('created_at').defaultNow().notNull(),
updatedAt: timestamp('updated_at').defaultNow().notNull(),
});

View File

@@ -8,6 +8,7 @@ export const ingestionProviderEnum = pgEnum('ingestion_provider', [
'generic_imap',
'pst_import',
'eml_import',
'mbox_import',
]);
export const ingestionStatusEnum = pgEnum('ingestion_status', [

View File

@@ -95,10 +95,7 @@ app.use('/v1/upload', uploadRouter);
// Middleware for all other routes
app.use((req, res, next) => {
// exclude certain API endpoints from the rate limiter, for example status, system settings
const excludedPatterns = [
/^\/v\d+\/auth\/status$/,
/^\/v\d+\/settings\/system$/
];
const excludedPatterns = [/^\/v\d+\/auth\/status$/, /^\/v\d+\/settings\/system$/];
for (const pattern of excludedPatterns) {
if (pattern.test(req.path)) {
return next();

View File

@@ -6,67 +6,67 @@ import { and, eq } from 'drizzle-orm';
import { ApiKey } from '@open-archiver/types';
export class ApiKeyService {
public static async generate(
userId: string,
name: string,
expiresInDays: number
): Promise<string> {
const key = randomBytes(32).toString('hex');
const expiresAt = new Date();
expiresAt.setDate(expiresAt.getDate() + expiresInDays);
const keyHash = createHash('sha256').update(key).digest('hex');
public static async generate(
userId: string,
name: string,
expiresInDays: number
): Promise<string> {
const key = randomBytes(32).toString('hex');
const expiresAt = new Date();
expiresAt.setDate(expiresAt.getDate() + expiresInDays);
const keyHash = createHash('sha256').update(key).digest('hex');
await db.insert(apiKeys).values({
userId,
name,
key: CryptoService.encrypt(key),
keyHash,
expiresAt
});
await db.insert(apiKeys).values({
userId,
name,
key: CryptoService.encrypt(key),
keyHash,
expiresAt,
});
return key;
}
return key;
}
public static async getKeys(userId: string): Promise<ApiKey[]> {
const keys = await db.select().from(apiKeys).where(eq(apiKeys.userId, userId));
public static async getKeys(userId: string): Promise<ApiKey[]> {
const keys = await db.select().from(apiKeys).where(eq(apiKeys.userId, userId));
return keys
.map((apiKey) => {
const decryptedKey = CryptoService.decrypt(apiKey.key);
if (!decryptedKey) {
return null;
}
return {
...apiKey,
key: decryptedKey.slice(0, 5) + "*****",
expiresAt: apiKey.expiresAt.toISOString(),
createdAt: apiKey.createdAt.toISOString()
};
})
.filter((k): k is NonNullable<typeof k> => k !== null);
}
return keys
.map((apiKey) => {
const decryptedKey = CryptoService.decrypt(apiKey.key);
if (!decryptedKey) {
return null;
}
return {
...apiKey,
key: decryptedKey.slice(0, 5) + '*****',
expiresAt: apiKey.expiresAt.toISOString(),
createdAt: apiKey.createdAt.toISOString(),
};
})
.filter((k): k is NonNullable<typeof k> => k !== null);
}
public static async deleteKey(id: string, userId: string) {
await db.delete(apiKeys).where(and(eq(apiKeys.id, id), eq(apiKeys.userId, userId)));
}
/**
*
* @param key API key
* @returns The owner user ID or null. null means the API key is not found.
*/
public static async validateKey(key: string): Promise<string | null> {
const keyHash = createHash('sha256').update(key).digest('hex');
const [apiKey] = await db.select().from(apiKeys).where(eq(apiKeys.keyHash, keyHash));
if (!apiKey || apiKey.expiresAt < new Date()) {
return null;
}
public static async deleteKey(id: string, userId: string) {
await db.delete(apiKeys).where(and(eq(apiKeys.id, id), eq(apiKeys.userId, userId)));
}
/**
*
* @param key API key
* @returns The owner user ID or null. null means the API key is not found.
*/
public static async validateKey(key: string): Promise<string | null> {
const keyHash = createHash('sha256').update(key).digest('hex');
const [apiKey] = await db.select().from(apiKeys).where(eq(apiKeys.keyHash, keyHash));
if (!apiKey || apiKey.expiresAt < new Date()) {
return null;
}
const decryptedKey = CryptoService.decrypt(apiKey.key);
if (decryptedKey !== key) {
// This should not happen if the hash matches, but as a security measure, we double-check.
return null;
}
const decryptedKey = CryptoService.decrypt(apiKey.key);
if (decryptedKey !== key) {
// This should not happen if the hash matches, but as a security measure, we double-check.
return null;
}
return apiKey.userId;
}
return apiKey.userId;
}
}

View File

@@ -5,6 +5,7 @@ import type {
GenericImapCredentials,
PSTImportCredentials,
EMLImportCredentials,
MboxImportCredentials,
EmailObject,
SyncState,
MailboxUser,
@@ -14,6 +15,7 @@ import { MicrosoftConnector } from './ingestion-connectors/MicrosoftConnector';
import { ImapConnector } from './ingestion-connectors/ImapConnector';
import { PSTConnector } from './ingestion-connectors/PSTConnector';
import { EMLConnector } from './ingestion-connectors/EMLConnector';
import { MboxConnector } from './ingestion-connectors/MboxConnector';
// Define a common interface for all connectors
export interface IEmailConnector {
@@ -43,6 +45,8 @@ export class EmailProviderFactory {
return new PSTConnector(credentials as PSTImportCredentials);
case 'eml_import':
return new EMLConnector(credentials as EMLImportCredentials);
case 'mbox_import':
return new MboxConnector(credentials as MboxImportCredentials);
default:
throw new Error(`Unsupported provider: ${source.provider}`);
}

View File

@@ -26,6 +26,7 @@ import { SearchService } from './SearchService';
import { DatabaseService } from './DatabaseService';
import { config } from '../config/index';
import { FilterBuilder } from './FilterBuilder';
import e from 'express';
export class IngestionService {
private static decryptSource(
@@ -47,7 +48,7 @@ export class IngestionService {
}
public static returnFileBasedIngestions(): IngestionProvider[] {
return ['pst_import', 'eml_import'];
return ['pst_import', 'eml_import', 'mbox_import'];
}
public static async create(
@@ -76,9 +77,13 @@ export class IngestionService {
const connector = EmailProviderFactory.createConnector(decryptedSource);
try {
await connector.testConnection();
const connectionValid = await connector.testConnection();
// If connection succeeds, update status to auth_success, which triggers the initial import.
return await this.update(decryptedSource.id, { status: 'auth_success' });
if (connectionValid) {
return await this.update(decryptedSource.id, { status: 'auth_success' });
} else {
throw Error('Ingestion authentication failed.')
}
} catch (error) {
// If connection fails, delete the newly created source and throw the error.
await this.delete(decryptedSource.id);

View File

@@ -69,7 +69,7 @@ export class EMLConnector implements IEmailConnector {
syncState?: SyncState | null
): AsyncGenerator<EmailObject | null> {
const fileStream = await this.storage.get(this.credentials.uploadedFilePath);
const tempDir = await fs.mkdtemp(join('/tmp', 'eml-import-'));
const tempDir = await fs.mkdtemp(join('/tmp', `eml-import-${new Date().getTime()}`));
const unzippedPath = join(tempDir, 'unzipped');
await fs.mkdir(unzippedPath);
const zipFilePath = join(tempDir, 'eml.zip');
@@ -115,6 +115,14 @@ export class EMLConnector implements IEmailConnector {
throw error;
} finally {
await fs.rm(tempDir, { recursive: true, force: true });
try {
await this.storage.delete(this.credentials.uploadedFilePath);
} catch (error) {
logger.error(
{ error, file: this.credentials.uploadedFilePath },
'Failed to delete EML file after processing.'
);
}
}
}

View File

@@ -0,0 +1,174 @@
import type {
MboxImportCredentials,
EmailObject,
EmailAddress,
SyncState,
MailboxUser,
} from '@open-archiver/types';
import type { IEmailConnector } from '../EmailProviderFactory';
import { simpleParser, ParsedMail, Attachment, AddressObject } from 'mailparser';
import { logger } from '../../config/logger';
import { getThreadId } from './helpers/utils';
import { StorageService } from '../StorageService';
import { Readable } from 'stream';
import { createHash } from 'crypto';
import { streamToBuffer } from '../../helpers/streamToBuffer';
export class MboxConnector implements IEmailConnector {
private storage: StorageService;
constructor(private credentials: MboxImportCredentials) {
this.storage = new StorageService();
}
public async testConnection(): Promise<boolean> {
try {
if (!this.credentials.uploadedFilePath) {
throw Error('Mbox file path not provided.');
}
if (!this.credentials.uploadedFilePath.includes('.mbox')) {
throw Error('Provided file is not in the MBOX format.');
}
const fileExist = await this.storage.exists(this.credentials.uploadedFilePath);
if (!fileExist) {
throw Error('Mbox file upload not finished yet, please wait.');
}
return true;
} catch (error) {
logger.error({ error, credentials: this.credentials }, 'Mbox file validation failed.');
throw error;
}
}
public async *listAllUsers(): AsyncGenerator<MailboxUser> {
const displayName =
this.credentials.uploadedFileName || `mbox-import-${new Date().getTime()}`;
logger.info(`Found potential mailbox: ${displayName}`);
const constructedPrimaryEmail = `${displayName.replace(/ /g, '.').toLowerCase()}@mbox.local`;
yield {
id: constructedPrimaryEmail,
primaryEmail: constructedPrimaryEmail,
displayName: displayName,
};
}
public async *fetchEmails(
userEmail: string,
syncState?: SyncState | null
): AsyncGenerator<EmailObject | null> {
try {
const fileStream = await this.storage.get(this.credentials.uploadedFilePath);
const fileBuffer = await streamToBuffer(fileStream as Readable);
const mboxContent = fileBuffer.toString('utf-8');
const emailDelimiter = '\nFrom ';
const emails = mboxContent.split(emailDelimiter);
// The first split part might be empty or part of the first email's header, so we adjust.
if (emails.length > 0 && !mboxContent.startsWith('From ')) {
emails.shift(); // Adjust if the file doesn't start with "From "
}
logger.info(`Found ${emails.length} potential emails in the mbox file.`);
let emailCount = 0;
for (const email of emails) {
try {
// Re-add the "From " delimiter for the parser, except for the very first email
const emailWithDelimiter =
emailCount > 0 || mboxContent.startsWith('From ') ? `From ${email}` : email;
const emailBuffer = Buffer.from(emailWithDelimiter, 'utf-8');
const emailObject = await this.parseMessage(emailBuffer, '');
yield emailObject;
emailCount++;
} catch (error) {
logger.error(
{ error, file: this.credentials.uploadedFilePath },
'Failed to process a single message from mbox file. Skipping.'
);
}
}
logger.info(`Finished processing mbox file. Total emails processed: ${emailCount}`);
} finally {
try {
await this.storage.delete(this.credentials.uploadedFilePath);
} catch (error) {
logger.error(
{ error, file: this.credentials.uploadedFilePath },
'Failed to delete mbox file after processing.'
);
}
}
}
private async parseMessage(emlBuffer: Buffer, path: string): Promise<EmailObject> {
const parsedEmail: ParsedMail = await simpleParser(emlBuffer);
const attachments = parsedEmail.attachments.map((attachment: Attachment) => ({
filename: attachment.filename || 'untitled',
contentType: attachment.contentType,
size: attachment.size,
content: attachment.content as Buffer,
}));
const mapAddresses = (
addresses: AddressObject | AddressObject[] | undefined
): EmailAddress[] => {
if (!addresses) return [];
const addressArray = Array.isArray(addresses) ? addresses : [addresses];
return addressArray.flatMap((a) =>
a.value.map((v) => ({
name: v.name,
address: v.address?.replaceAll(`'`, '') || '',
}))
);
};
const threadId = getThreadId(parsedEmail.headers);
let messageId = parsedEmail.messageId;
if (!messageId) {
messageId = `generated-${createHash('sha256').update(emlBuffer).digest('hex')}`;
}
const from = mapAddresses(parsedEmail.from);
if (from.length === 0) {
from.push({ name: 'No Sender', address: 'No Sender' });
}
// Extract folder path from headers. Mbox files don't have a standard folder structure, so we rely on custom headers added by email clients.
// Gmail uses 'X-Gmail-Labels', and other clients like Thunderbird may use 'X-Folder'.
const gmailLabels = parsedEmail.headers.get('x-gmail-labels');
const folderHeader = parsedEmail.headers.get('x-folder');
let finalPath = '';
if (gmailLabels && typeof gmailLabels === 'string') {
// We take the first label as the primary folder.
// Gmail labels can be hierarchical, but we'll simplify to the first label.
finalPath = gmailLabels.split(',')[0];
} else if (folderHeader && typeof folderHeader === 'string') {
finalPath = folderHeader;
}
return {
id: messageId,
threadId: threadId,
from,
to: mapAddresses(parsedEmail.to),
cc: mapAddresses(parsedEmail.cc),
bcc: mapAddresses(parsedEmail.bcc),
subject: parsedEmail.subject || '',
body: parsedEmail.text || '',
html: parsedEmail.html || '',
headers: parsedEmail.headers,
attachments,
receivedAt: parsedEmail.date || new Date(),
eml: emlBuffer,
path: finalPath,
};
}
public getUpdatedSyncState(): SyncState {
return {};
}
}

View File

@@ -193,6 +193,14 @@ export class PSTConnector implements IEmailConnector {
throw error;
} finally {
pstFile?.close();
try {
await this.storage.delete(this.credentials.uploadedFilePath);
} catch (error) {
logger.error(
{ error, file: this.credentials.uploadedFilePath },
'Failed to delete PST file after processing.'
);
}
}
}
@@ -273,8 +281,8 @@ export class PSTConnector implements IEmailConnector {
emlBuffer ?? Buffer.from(parsedEmail.text || parsedEmail.html || '', 'utf-8')
)
.digest('hex')}-${createHash('sha256')
.update(emlBuffer ?? Buffer.from(msg.subject || '', 'utf-8'))
.digest('hex')}-${msg.clientSubmitTime?.getTime()}`;
.update(emlBuffer ?? Buffer.from(msg.subject || '', 'utf-8'))
.digest('hex')}-${msg.clientSubmitTime?.getTime()}`;
}
return {
id: messageId,

View File

@@ -15,13 +15,15 @@
"dependencies": {
"@iconify/svelte": "^5.0.1",
"@open-archiver/types": "workspace:*",
"@sveltejs/kit": "^2.16.0",
"@sveltejs/kit": "^2.38.1",
"bits-ui": "^2.8.10",
"clsx": "^2.1.1",
"d3-shape": "^3.2.0",
"html-entities": "^2.6.0",
"jose": "^6.0.1",
"lucide-svelte": "^0.525.0",
"postal-mime": "^2.4.4",
"semver": "^7.7.2",
"svelte-persisted-store": "^0.12.0",
"sveltekit-i18n": "^2.4.2",
"tailwind-merge": "^3.3.1",
@@ -35,6 +37,7 @@
"@sveltejs/vite-plugin-svelte": "^5.0.0",
"@tailwindcss/vite": "^4.0.0",
"@types/d3-shape": "^3.1.7",
"@types/semver": "^7.7.1",
"dotenv": "^17.2.0",
"layerchart": "2.0.0-next.27",
"mode-watcher": "^1.1.0",

View File

@@ -2,6 +2,7 @@
import PostalMime, { type Email } from 'postal-mime';
import type { Buffer } from 'buffer';
import { t } from '$lib/translations';
import { encode } from 'html-entities';
let {
raw,
@@ -18,7 +19,9 @@
if (parsedEmail && parsedEmail.html) {
return `<base target="_blank" />${parsedEmail.html}`;
} else if (parsedEmail && parsedEmail.text) {
return `<base target="_blank" />${parsedEmail.text}`;
// display raw text email body in html
const safeHtmlContent: string = encode(parsedEmail.text);
return `<base target="_blank" /><div>${safeHtmlContent.replaceAll('\n', '<br>')}</div>`;
} else if (rawHtml) {
return `<base target="_blank" />${rawHtml}`;
}
@@ -52,16 +55,16 @@
<div class="mt-2 rounded-md border bg-white p-4">
{#if isLoading}
<p>{$t('components.email_preview.loading')}</p>
{:else if emailHtml}
<p>{$t('app.components.email_preview.loading')}</p>
{:else if emailHtml()}
<iframe
title={$t('archive.email_preview')}
title={$t('app.archive.email_preview')}
srcdoc={emailHtml()}
class="h-[600px] w-full border-none"
></iframe>
{:else if raw}
<p>{$t('components.email_preview.render_error')}</p>
<p>{$t('app.components.email_preview.render_error')}</p>
{:else}
<p class="text-gray-500">{$t('components.email_preview.not_available')}</p>
<p class="text-gray-500">{$t('app.components.email_preview.not_available')}</p>
{/if}
</div>

View File

@@ -1,18 +1,39 @@
<script lang="ts">
import { t } from '$lib/translations';
import * as Alert from '$lib/components/ui/alert';
import { Info } from 'lucide-svelte';
export let currentVersion: string;
export let newVersionInfo: { version: string; description: string; url: string } | null = null;
</script>
<footer class="bg-muted py-6 md:py-0">
<div
class="container mx-auto flex flex-col items-center justify-center gap-4 md:h-24 md:flex-row"
>
<div class="container mx-auto flex flex-col items-center justify-center gap-4 py-8 md:flex-row">
<div class="flex flex-col items-center gap-2">
{#if newVersionInfo}
<Alert.Root>
<Alert.Title class="flex items-center gap-2">
<Info class="h-4 w-4" />
{$t('app.components.footer.new_version_available')}
<a
href={newVersionInfo.url}
target="_blank"
class=" text-muted-foreground underline"
>
{newVersionInfo.description}
</a>
</Alert.Title>
</Alert.Root>
{/if}
<p class="text-balance text-center text-xs font-medium leading-loose">
© {new Date().getFullYear()}
<a href="https://openarchiver.com/" target="_blank">Open Archiver</a>. {$t(
'app.components.footer.all_rights_reserved'
)}
</p>
<p class="text-balance text-center text-xs font-medium leading-loose">
Version: {currentVersion}
</p>
</div>
</div>
</footer>

View File

@@ -41,6 +41,10 @@
value: 'eml_import',
label: $t('app.components.ingestion_source_form.provider_eml_import'),
},
{
value: 'mbox_import',
label: $t('app.components.ingestion_source_form.provider_mbox_import'),
},
];
let formData: CreateIngestionSourceDto = $state({
@@ -55,7 +59,6 @@
$effect(() => {
formData.providerConfig.type = formData.provider;
console.log(formData);
});
const triggerContent = $derived(
@@ -101,7 +104,6 @@
formData.providerConfig.uploadedFilePath = result.filePath;
formData.providerConfig.uploadedFileName = file.name;
fileUploading = false;
} catch (error) {
fileUploading = false;
@@ -224,10 +226,13 @@
<Checkbox id="secure" bind:checked={formData.providerConfig.secure} />
</div>
<div class="grid grid-cols-4 items-center gap-4">
<Label for="secure" class="text-left"
<Label for="allowInsecureCert" class="text-left"
>{$t('app.components.ingestion_source_form.allow_insecure_cert')}</Label
>
<Checkbox id="secure" bind:checked={formData.providerConfig.allowInsecureCert} />
<Checkbox
id="allowInsecureCert"
bind:checked={formData.providerConfig.allowInsecureCert}
/>
</div>
{:else if formData.provider === 'pst_import'}
<div class="grid grid-cols-4 items-center gap-4">
@@ -265,6 +270,24 @@
{/if}
</div>
</div>
{:else if formData.provider === 'mbox_import'}
<div class="grid grid-cols-4 items-center gap-4">
<Label for="mbox-file" class="text-left"
>{$t('app.components.ingestion_source_form.mbox_file')}</Label
>
<div class="col-span-3 flex flex-row items-center space-x-2">
<Input
id="mbox-file"
type="file"
class=""
accept=".mbox"
onchange={handleFileChange}
/>
{#if fileUploading}
<span class=" text-primary animate-spin"><Loader2 /></span>
{/if}
</div>
</div>
{/if}
{#if formData.provider === 'google_workspace' || formData.provider === 'microsoft_365'}
<Alert.Root>

View File

@@ -163,7 +163,8 @@
"not_available": "Raw .eml file not available for this email."
},
"footer": {
"all_rights_reserved": "All rights reserved."
"all_rights_reserved": "All rights reserved.",
"new_version_available": "New version available"
},
"ingestion_source_form": {
"provider_generic_imap": "Generic IMAP",
@@ -171,6 +172,7 @@
"provider_microsoft_365": "Microsoft 365",
"provider_pst_import": "PST Import",
"provider_eml_import": "EML Import",
"provider_mbox_import": "Mbox Import",
"select_provider": "Select a provider",
"service_account_key": "Service Account Key (JSON)",
"service_account_key_placeholder": "Paste your service account key JSON content",
@@ -186,6 +188,7 @@
"allow_insecure_cert": "Allow insecure cert",
"pst_file": "PST File",
"eml_file": "EML File",
"mbox_file": "Mbox File",
"heads_up": "Heads up!",
"org_wide_warning": "Please note that this is an organization-wide operation. This kind of ingestions will import and index <b>all</b> email inboxes in your organization. If you want to import only specific email inboxes, use the IMAP connector.",
"upload_failed": "Upload Failed, please try again"

View File

@@ -1,17 +1,17 @@
{
"app": {
"auth": {
"login": "Accesso",
"login": "Accedi",
"login_tip": "Inserisci la tua email qui sotto per accedere al tuo account.",
"email": "Email",
"password": "Password"
},
"common": {
"working": "In lavorazione"
"working": "In corso"
},
"archive": {
"title": "Archivio",
"no_subject": "Nessun oggetto",
"no_subject": "Nessun Oggetto",
"from": "Da",
"sent": "Inviato",
"recipients": "Destinatari",
@@ -20,27 +20,27 @@
"folder": "Cartella",
"tags": "Tag",
"size": "Dimensione",
"email_preview": "Anteprima email",
"email_preview": "Anteprima Email",
"attachments": "Allegati",
"download": "Scarica",
"actions": "Azioni",
"download_eml": "Scarica email (.eml)",
"delete_email": "Elimina email",
"email_thread": "Thread email",
"download_eml": "Scarica Email (.eml)",
"delete_email": "Elimina Email",
"email_thread": "Thread Email",
"delete_confirmation_title": "Sei sicuro di voler eliminare questa email?",
"delete_confirmation_description": "Questa azione non può essere annullata ed eliminerà permanentemente l'email e i suoi allegati.",
"delete_confirmation_description": "Questa azione non può essere annullata e rimuoverà permanentemente l'email e i suoi allegati.",
"deleting": "Eliminazione in corso",
"confirm": "Conferma",
"cancel": "Annulla",
"not_found": "Email non trovata."
},
"ingestions": {
"title": "Fonti di ingestione",
"ingestion_sources": "Fonti di ingestione",
"bulk_actions": "Azioni di massa",
"force_sync": "Forza sincronizzazione",
"title": "Sorgenti di Ingestione",
"ingestion_sources": "Sorgenti di Ingestione",
"bulk_actions": "Azioni di Massa",
"force_sync": "Forza Sincronizzazione",
"delete": "Elimina",
"create_new": "Crea nuovo",
"create_new": "Crea Nuovo",
"name": "Nome",
"provider": "Provider",
"status": "Stato",
@@ -52,28 +52,28 @@
"open_menu": "Apri menu",
"edit": "Modifica",
"create": "Crea",
"ingestion_source": "Fonte di ingestione",
"edit_description": "Apporta modifiche alla tua fonte di ingestione qui.",
"create_description": "Aggiungi una nuova fonte di ingestione per iniziare ad archiviare le email.",
"ingestion_source": "Sorgente di Ingestione",
"edit_description": "Apporta modifiche alla tua sorgente di ingestione qui.",
"create_description": "Aggiungi una nuova sorgente di ingestione per iniziare ad archiviare le email.",
"read": "Leggi",
"docs_here": "documenti qui",
"delete_confirmation_title": "Sei sicuro di voler eliminare questa ingestione?",
"delete_confirmation_description": "Questo eliminerà tutte le email archiviate, gli allegati, l'indicizzazione e i file associati a questa ingestione. Se desideri solo interrompere la sincronizzazione di nuove email, puoi invece mettere in pausa l'ingestione.",
"delete_confirmation_description": "Questo cancellerà tutte le email archiviate, gli allegati, l'indicizzazione e i file associati a questa ingestione. Se vuoi solo interrompere la sincronizzazione di nuove email, puoi mettere in pausa l'ingestione.",
"deleting": "Eliminazione in corso",
"confirm": "Conferma",
"cancel": "Annulla",
"bulk_delete_confirmation_title": "Sei sicuro di voler eliminare {{count}} ingestioni selezionate?",
"bulk_delete_confirmation_description": "Questo eliminerà tutte le email archiviate, gli allegati, l'indicizzazione e i file associati a queste ingestioni. Se desideri solo interrompere la sincronizzazione di nuove email, puoi invece mettere in pausa le ingestioni."
"bulk_delete_confirmation_description": "Questo cancellerà tutte le email archiviate, gli allegati, l'indicizzazione e i file associati a queste ingestioni. Se vuoi solo interrompere la sincronizzazione di nuove email, puoi mettere in pausa le ingestioni."
},
"search": {
"title": "Cerca",
"description": "Cerca email archiviate.",
"email_search": "Ricerca email",
"title": "Ricerca",
"description": "Ricerca email archiviate.",
"email_search": "Ricerca Email",
"placeholder": "Cerca per parola chiave, mittente, destinatario...",
"search_button": "Cerca",
"search_options": "Opzioni di ricerca",
"strategy_fuzzy": "Fuzzy",
"strategy_verbatim": "Verbatim",
"strategy_fuzzy": "Approssimativa",
"strategy_verbatim": "Esatta",
"strategy_frequency": "Frequenza",
"select_strategy": "Seleziona una strategia",
"error": "Errore",
@@ -87,18 +87,18 @@
"next": "Succ"
},
"roles": {
"title": "Gestione ruoli",
"role_management": "Gestione ruoli",
"create_new": "Crea nuovo",
"title": "Gestione Ruoli",
"role_management": "Gestione Ruoli",
"create_new": "Crea Nuovo",
"name": "Nome",
"created_at": "Creato il",
"actions": "Azioni",
"open_menu": "Apri menu",
"view_policy": "Visualizza policy",
"view_policy": "Visualizza Policy",
"edit": "Modifica",
"delete": "Elimina",
"no_roles_found": "Nessun ruolo trovato.",
"role_policy": "Policy ruolo",
"role_policy": "Policy Ruolo",
"viewing_policy_for_role": "Visualizzazione policy per il ruolo: {{name}}",
"create": "Crea",
"role": "Ruolo",
@@ -111,22 +111,22 @@
"cancel": "Annulla"
},
"system_settings": {
"title": "Impostazioni di sistema",
"system_settings": "Impostazioni di sistema",
"title": "Impostazioni di Sistema",
"system_settings": "Impostazioni di Sistema",
"description": "Gestisci le impostazioni globali dell'applicazione.",
"language": "Lingua",
"default_theme": "Tema predefinito",
"light": "Chiaro",
"dark": "Scuro",
"system": "Sistema",
"support_email": "Email di supporto",
"saving": "Salvataggio",
"save_changes": "Salva modifiche"
"support_email": "Email di Supporto",
"saving": "Salvataggio in corso",
"save_changes": "Salva Modifiche"
},
"users": {
"title": "Gestione utenti",
"user_management": "Gestione utenti",
"create_new": "Crea nuovo",
"title": "Gestione Utenti",
"user_management": "Gestione Utenti",
"create_new": "Crea Nuovo",
"name": "Nome",
"email": "Email",
"role": "Ruolo",
@@ -146,33 +146,10 @@
"confirm": "Conferma",
"cancel": "Annulla"
},
"setup": {
"title": "Configurazione",
"description": "Configura l'account amministratore iniziale per Open Archiver.",
"welcome": "Benvenuto",
"create_admin_account": "Crea il primo account amministratore per iniziare.",
"first_name": "Nome",
"last_name": "Cognome",
"email": "Email",
"password": "Password",
"creating_account": "Creazione account",
"create_account": "Crea account"
},
"layout": {
"dashboard": "Dashboard",
"ingestions": "Ingestioni",
"archived_emails": "Email archiviate",
"search": "Cerca",
"settings": "Impostazioni",
"system": "Sistema",
"users": "Utenti",
"roles": "Ruoli",
"logout": "Esci"
},
"components": {
"charts": {
"emails_ingested": "Email ingerite",
"storage_used": "Spazio di archiviazione utilizzato",
"emails_ingested": "Email Acquisite",
"storage_used": "Spazio di Archiviazione Utilizzato",
"emails": "Email"
},
"common": {
@@ -182,35 +159,36 @@
},
"email_preview": {
"loading": "Caricamento anteprima email...",
"render_error": "Impossibile visualizzare l'anteprima dell'email.",
"not_available": "File .eml non disponibile per questa email."
"render_error": "Impossibile renderizzare l'anteprima dell'email.",
"not_available": "File .eml grezzo non disponibile per questa email."
},
"footer": {
"all_rights_reserved": "Tutti i diritti riservati."
},
"ingestion_source_form": {
"provider_generic_imap": "IMAP generico",
"provider_generic_imap": "IMAP Generico",
"provider_google_workspace": "Google Workspace",
"provider_microsoft_365": "Microsoft 365",
"provider_pst_import": "Importazione PST",
"provider_eml_import": "Importazione EML",
"select_provider": "Seleziona un provider",
"service_account_key": "Chiave account di servizio (JSON)",
"service_account_key_placeholder": "Incolla il contenuto JSON della tua chiave account di servizio",
"impersonated_admin_email": "Email amministratore impersonata",
"client_id": "ID applicazione (client)",
"client_secret": "Valore segreto client",
"client_secret_placeholder": "Inserisci il valore segreto, non l'ID segreto",
"tenant_id": "ID directory (tenant)",
"service_account_key": "Chiave Account di Servizio (JSON)",
"service_account_key_placeholder": "Incolla il contenuto JSON della chiave del tuo account di servizio",
"impersonated_admin_email": "Email dell'Amministratore Impersonato",
"client_id": "ID Applicazione (Client)",
"client_secret": "Valore Segreto Client",
"client_secret_placeholder": "Inserisci il Valore segreto, non l'ID Segreto",
"tenant_id": "ID Directory (Tenant)",
"host": "Host",
"port": "Porta",
"username": "Nome utente",
"username": "Nome Utente",
"use_tls": "Usa TLS",
"allow_insecure_cert": "Consenti certificato non sicuro",
"pst_file": "File PST",
"eml_file": "File EML",
"heads_up": "Attenzione!",
"org_wide_warning": "Si prega di notare che questa è un'operazione a livello di organizzazione. Questo tipo di ingestione importerà e indicizzerà <b>tutte</b> le caselle di posta elettronica della tua organizzazione. Se desideri importare solo caselle di posta elettronica specifiche, utilizza il connettore IMAP.",
"upload_failed": "Caricamento non riuscito, riprova"
"org_wide_warning": "Si prega di notare che questa è un'operazione a livello di organizzazione. Questo tipo di ingestione importerà e indicizzerà <b>tutte</b> le caselle di posta elettronica nella tua organizzazione. Se vuoi importare solo caselle di posta elettronica specifiche, usa il connettore IMAP.",
"upload_failed": "Caricamento Fallito, riprova"
},
"role_form": {
"policies_json": "Policy (JSON)",
@@ -223,28 +201,61 @@
"select_role": "Seleziona un ruolo"
}
},
"dashboard_page": {
"title": "Dashboard",
"meta_description": "Panoramica del tuo archivio email.",
"header": "Dashboard",
"create_ingestion": "Crea un'ingestione",
"no_ingestion_header": "Non hai alcuna fonte di ingestione configurata.",
"no_ingestion_text": "Aggiungi una fonte di ingestione per iniziare ad archiviare le tue caselle di posta.",
"total_emails_archived": "Email totali archiviate",
"total_storage_used": "Spazio di archiviazione totale utilizzato",
"failed_ingestions": "Ingestioni non riuscite (ultimi 7 giorni)",
"ingestion_history": "Cronologia ingestioni",
"no_ingestion_history": "Nessuna cronologia di ingestione disponibile.",
"storage_by_source": "Archiviazione per fonte di ingestione",
"no_ingestion_sources": "Nessuna fonte di ingestione disponibile.",
"indexed_insights": "Approfondimenti indicizzati",
"top_10_senders": "Top 10 mittenti",
"no_indexed_insights": "Nessun approfondimento indicizzato disponibile."
"setup": {
"title": "Configurazione",
"description": "Configura l'account amministratore iniziale per Open Archiver.",
"welcome": "Benvenuto",
"create_admin_account": "Crea il primo account amministratore per iniziare.",
"first_name": "Nome",
"last_name": "Cognome",
"email": "Email",
"password": "Password",
"creating_account": "Creazione Account",
"create_account": "Crea Account"
},
"layout": {
"dashboard": "Dashboard",
"ingestions": "Ingestioni",
"archived_emails": "Email archiviate",
"search": "Ricerca",
"settings": "Impostazioni",
"system": "Sistema",
"users": "Utenti",
"roles": "Ruoli",
"api_keys": "Chiavi API",
"logout": "Esci"
},
"api_keys_page": {
"title": "Chiavi API",
"header": "Chiavi API",
"generate_new_key": "Genera Nuova Chiave",
"name": "Nome",
"key": "Chiave",
"expires_at": "Scade il",
"created_at": "Creato il",
"actions": "Azioni",
"delete": "Elimina",
"no_keys_found": "Nessuna chiave API trovata.",
"generate_modal_title": "Genera Nuova Chiave API",
"generate_modal_description": "Fornisci un nome e una scadenza per la tua nuova chiave API.",
"expires_in": "Scade Tra",
"select_expiration": "Seleziona una scadenza",
"30_days": "30 Giorni",
"60_days": "60 Giorni",
"6_months": "6 Mesi",
"12_months": "12 Mesi",
"24_months": "24 Mesi",
"generate": "Genera",
"new_api_key": "Nuova Chiave API",
"failed_to_delete": "Impossibile eliminare la chiave API",
"api_key_deleted": "Chiave API eliminata",
"generated_title": "Chiave API Generata",
"generated_message": "La tua chiave API è stata generata, per favore copiala e salvala in un luogo sicuro. Questa chiave verrà mostrata solo una volta."
},
"archived_emails_page": {
"title": "Email archiviate",
"header": "Email archiviate",
"select_ingestion_source": "Seleziona una fonte di ingestione",
"header": "Email Archiviate",
"select_ingestion_source": "Seleziona una sorgente di ingestione",
"date": "Data",
"subject": "Oggetto",
"sender": "Mittente",
@@ -255,6 +266,24 @@
"no_emails_found": "Nessuna email archiviata trovata.",
"prev": "Prec",
"next": "Succ"
},
"dashboard_page": {
"title": "Dashboard",
"meta_description": "Panoramica del tuo archivio email.",
"header": "Dashboard",
"create_ingestion": "Crea un'ingestione",
"no_ingestion_header": "Non hai impostato nessuna sorgente di ingestione.",
"no_ingestion_text": "Aggiungi una sorgente di ingestione per iniziare ad archiviare le tue caselle di posta.",
"total_emails_archived": "Totale Email Archiviate",
"total_storage_used": "Spazio di Archiviazione Totale Utilizzato",
"failed_ingestions": "Ingestioni Fallite (Ultimi 7 Giorni)",
"ingestion_history": "Cronologia Ingestioni",
"no_ingestion_history": "Nessuna cronologia delle ingestioni disponibile.",
"storage_by_source": "Spazio di Archiviazione per Sorgente di Ingestione",
"no_ingestion_sources": "Nessuna sorgente di ingestione disponibile.",
"indexed_insights": "Approfondimenti indicizzati",
"top_10_senders": "I 10 Mittenti Principali",
"no_indexed_insights": "Nessun approfondimento indicizzato disponibile."
}
}
}

View File

@@ -3,6 +3,11 @@ import type { LayoutServerLoad } from './$types';
import 'dotenv/config';
import { api } from '$lib/server/api';
import type { SystemSettings } from '@open-archiver/types';
import { version } from '../../../../package.json';
import semver from 'semver';
let newVersionInfo: { version: string; description: string; url: string } | null = null;
let lastChecked: Date | null = null;
export const load: LayoutServerLoad = async (event) => {
const { locals, url } = event;
@@ -32,10 +37,35 @@ export const load: LayoutServerLoad = async (event) => {
? await systemSettingsResponse.json()
: null;
const now = new Date();
if (!lastChecked || now.getTime() - lastChecked.getTime() > 1000 * 60 * 60) {
try {
const res = await fetch(
'https://api.github.com/repos/LogicLabs-OU/OpenArchiver/releases/latest'
);
if (res.ok) {
const latestRelease = await res.json();
const latestVersion = latestRelease.tag_name.replace('v', '');
if (semver.gt(latestVersion, version)) {
newVersionInfo = {
version: latestVersion,
description: latestRelease.name,
url: latestRelease.html_url,
};
}
}
lastChecked = now;
} catch (error) {
console.error('Failed to fetch latest version from GitHub:', error);
}
}
return {
user: locals.user,
accessToken: locals.accessToken,
isDemo: process.env.IS_DEMO === 'true',
systemSettings,
currentVersion: version,
newVersionInfo: newVersionInfo,
};
};

View File

@@ -35,5 +35,5 @@
<main class="flex-1">
{@render children()}
</main>
<Footer />
<Footer currentVersion={data.currentVersion} newVersionInfo={data.newVersionInfo} />
</div>

View File

@@ -435,7 +435,12 @@
</div>
<Dialog.Root bind:open={isDialogOpen}>
<Dialog.Content class="sm:max-w-120 md:max-w-180">
<Dialog.Content
class="sm:max-w-120 md:max-w-180"
onInteractOutside={(e) => {
e.preventDefault();
}}
>
<Dialog.Header>
<Dialog.Title
>{selectedSource ? $t('app.ingestions.edit') : $t('app.ingestions.create')}{' '}

View File

@@ -2,49 +2,48 @@ import { api } from '$lib/server/api';
import type { Actions, PageServerLoad } from './$types';
export const load: PageServerLoad = async (event) => {
const response = await api('/api-keys', event);
const apiKeys = await response.json();
const response = await api('/api-keys', event);
const apiKeys = await response.json();
return {
apiKeys,
};
return {
apiKeys,
};
};
export const actions: Actions = {
generate: async (event) => {
const data = await event.request.formData();
const name = data.get('name') as string;
const expiresInDays = Number(data.get('expiresInDays'));
generate: async (event) => {
const data = await event.request.formData();
const name = data.get('name') as string;
const expiresInDays = Number(data.get('expiresInDays'));
const response = await api('/api-keys', event, {
method: 'POST',
body: JSON.stringify({ name, expiresInDays }),
});
const response = await api('/api-keys', event, {
method: 'POST',
body: JSON.stringify({ name, expiresInDays }),
});
const responseBody = await response.json();
const responseBody = await response.json();
if (!response.ok) {
return {
message: responseBody.message || '',
errors: responseBody.errors
}
}
if (!response.ok) {
return {
message: responseBody.message || '',
errors: responseBody.errors,
};
}
return {
newApiKey: responseBody.key,
};
},
delete: async (event) => {
const data = await event.request.formData();
const id = data.get('id') as string;
return {
newApiKey: responseBody.key,
};
},
delete: async (event) => {
const data = await event.request.formData();
const id = data.get('id') as string;
await api(`/api-keys/${id}`, event, {
method: 'DELETE',
});
await api(`/api-keys/${id}`, event, {
method: 'DELETE',
});
return {
success: true,
};
},
return {
success: true,
};
},
};

View File

@@ -23,7 +23,8 @@ export type IngestionProvider =
| 'microsoft_365'
| 'generic_imap'
| 'pst_import'
| 'eml_import';
| 'eml_import'
| 'mbox_import';
export type IngestionStatus =
| 'active'
@@ -81,13 +82,20 @@ export interface EMLImportCredentials extends BaseIngestionCredentials {
uploadedFilePath: string;
}
export interface MboxImportCredentials extends BaseIngestionCredentials {
type: 'mbox_import';
uploadedFileName: string;
uploadedFilePath: string;
}
// Discriminated union for all possible credential types
export type IngestionCredentials =
| GenericImapCredentials
| GoogleWorkspaceCredentials
| Microsoft365Credentials
| PSTImportCredentials
| EMLImportCredentials;
| EMLImportCredentials
| MboxImportCredentials;
export interface IngestionSource {
id: string;

478
pnpm-lock.yaml generated

File diff suppressed because it is too large Load Diff