* fix(api): correct API key generation and proxy handling This commit resolves an issue where generating a new API key would fail. The root cause was improper handling of POST request bodies in the frontend proxy server. - Refactored `ApiKeyController` methods to use arrow functions to ensure correct `this` binding. * User profile/account page, change password, API * docs(api): update ingestion source provider values Update the `CreateIngestionSourceDto` documentation in `ingestion.md` to reflect the current set of supported providers. * updating tag * feat: add REDIS_USER env variable (#172) * feat: add REDIS_USER env variable fixes #171 * add proper type for bullmq config * Bulgarian UI language strings added (backend+frontend) (#194) * Bulgarian UI Support added * BG language UI support - Create translation.json * update redis config logic * Update Bulgarian language setting, register language * Allow specifying local file path for mbox/eml/pst (#214) * Add agents AI doc * Allow local file path for Mbox file ingestion --------- Co-authored-by: Wei S. <5291640+wayneshn@users.noreply.github.com> * feat(ingestion): add local file path support and optimize EML processing - Frontend: Updated IngestionSourceForm to allow toggling between "Upload File" and "Local File Path" for PST, EML, and Mbox providers. - Frontend: Added logic to clear irrelevant form data when switching import methods. - Frontend: Added English translations for new form fields. - Backend: Refactored EMLConnector to stream ZIP entries using yauzl instead of extracting the full archive to disk, significantly improving efficiency for large archives. - Docs: Updated API documentation and User Guides (PST, EML, Mbox) to clarify "Local File Path" usage, specifically within Docker environments. * docs: add meilisearch dumpless upgrade guide and snapshot config Update `docker-compose.yml` to include the `MEILI_SCHEDULE_SNAPSHOT` environment variable, defaulting to 86400 seconds (24 hours), enabling periodic data snapshots for easier recovery. Shout out to @morph027 for the inspiration! Additionally, update the Meilisearch upgrade documentation to include an experimental "dumpless" upgrade guide while marking the previous method as the standard recommended process. * build(coolify): enable daily snapshots for meilisearch Configure the Meilisearch service in `open-archiver.yml` to create snapshots every 86400 seconds (24 hours) by setting the `MEILI_SCHEDULE_SNAPSHOT` environment variable. --------- Co-authored-by: Antonia Schwennesen <53372671+zophiana@users.noreply.github.com> Co-authored-by: IT Creativity + Art Team <admin@it-playground.net> Co-authored-by: Jan Berdajs <mrbrdo@gmail.com>
5.7 KiB
Ingestion Service API
The Ingestion Service manages ingestion sources, which are configurations for connecting to email providers and importing emails.
Endpoints
All endpoints in this service require authentication.
POST /api/v1/ingestion-sources
Creates a new ingestion source.
Access: Authenticated
Request Body
The request body should be a CreateIngestionSourceDto object.
interface CreateIngestionSourceDto {
name: string;
provider: 'google_workspace' | 'microsoft_365' | 'generic_imap' | 'pst_import' | 'eml_import' | 'mbox_import';
providerConfig: IngestionCredentials;
}
Example: Creating an Mbox Import Source with File Upload
{
"name": "My Mbox Import",
"provider": "mbox_import",
"providerConfig": {
"type": "mbox_import",
"uploadedFileName": "emails.mbox",
"uploadedFilePath": "open-archiver/tmp/uuid-emails.mbox"
}
}
Example: Creating an Mbox Import Source with Local File Path
{
"name": "My Mbox Import",
"provider": "mbox_import",
"providerConfig": {
"type": "mbox_import",
"localFilePath": "/path/to/emails.mbox"
}
}
Note: When using localFilePath, the file will not be deleted after import. When using uploadedFilePath (via the upload API), the file will be automatically deleted after import. The same applies to pst_import and eml_import providers.
Important regarding localFilePath: When running OpenArchiver in a Docker container (which is the standard deployment), localFilePath refers to the path inside the Docker container, not on the host machine.
To use a local file:
- Recommended: Place your file inside the directory defined by
STORAGE_LOCAL_ROOT_PATH(e.g., inside atempfolder). Since this directory is already mounted as a volume, the file will be accessible at the same path inside the container. - Alternative: Mount a specific directory containing your files as a volume in
docker-compose.yml. For example, add- /path/to/my/files:/importsto thevolumessection and use/imports/myfile.pstas thelocalFilePath.
Responses
- 201 Created: The newly created ingestion source.
- 500 Internal Server Error: An unexpected error occurred.
GET /api/v1/ingestion-sources
Retrieves all ingestion sources.
Access: Authenticated
Responses
- 200 OK: An array of ingestion source objects.
- 500 Internal Server Error: An unexpected error occurred.
GET /api/v1/ingestion-sources/:id
Retrieves a single ingestion source by its ID.
Access: Authenticated
URL Parameters
| Parameter | Type | Description |
|---|---|---|
id |
string | The ID of the ingestion source. |
Responses
- 200 OK: The ingestion source object.
- 404 Not Found: Ingestion source not found.
- 500 Internal Server Error: An unexpected error occurred.
PUT /api/v1/ingestion-sources/:id
Updates an existing ingestion source.
Access: Authenticated
URL Parameters
| Parameter | Type | Description |
|---|---|---|
id |
string | The ID of the ingestion source. |
Request Body
The request body should be an UpdateIngestionSourceDto object.
interface UpdateIngestionSourceDto {
name?: string;
provider?: 'google' | 'microsoft' | 'generic_imap';
providerConfig?: IngestionCredentials;
status?: 'pending_auth' | 'auth_success' | 'importing' | 'active' | 'paused' | 'error';
}
Responses
- 200 OK: The updated ingestion source object.
- 404 Not Found: Ingestion source not found.
- 500 Internal Server Error: An unexpected error occurred.
DELETE /api/v1/ingestion-sources/:id
Deletes an ingestion source and all associated data.
Access: Authenticated
URL Parameters
| Parameter | Type | Description |
|---|---|---|
id |
string | The ID of the ingestion source. |
Responses
- 204 No Content: The ingestion source was deleted successfully.
- 404 Not Found: Ingestion source not found.
- 500 Internal Server Error: An unexpected error occurred.
POST /api/v1/ingestion-sources/:id/import
Triggers the initial import process for an ingestion source.
Access: Authenticated
URL Parameters
| Parameter | Type | Description |
|---|---|---|
id |
string | The ID of the ingestion source. |
Responses
- 202 Accepted: The initial import was triggered successfully.
- 404 Not Found: Ingestion source not found.
- 500 Internal Server Error: An unexpected error occurred.
POST /api/v1/ingestion-sources/:id/pause
Pauses an active ingestion source.
Access: Authenticated
URL Parameters
| Parameter | Type | Description |
|---|---|---|
id |
string | The ID of the ingestion source. |
Responses
- 200 OK: The updated ingestion source object with a
pausedstatus. - 404 Not Found: Ingestion source not found.
- 500 Internal Server Error: An unexpected error occurred.
POST /api/v1/ingestion-sources/:id/sync
Triggers a forced synchronization for an ingestion source.
Access: Authenticated
URL Parameters
| Parameter | Type | Description |
|---|---|---|
id |
string | The ID of the ingestion source. |
Responses
- 202 Accepted: The force sync was triggered successfully.
- 404 Not Found: Ingestion source not found.
- 500 Internal Server Error: An unexpected error occurred.