Files
OpenArchiver/docs/api/ingestion.md
Wei S. 7dac3b2bfd V0.4.2 (#310)
* fix(api): correct API key generation and proxy handling

This commit resolves an issue where generating a new API key would fail. The root cause was improper handling of POST request bodies in the frontend proxy server.

- Refactored `ApiKeyController` methods to use arrow functions to ensure correct `this` binding.

* User profile/account page, change password, API

* docs(api): update ingestion source provider values

Update the `CreateIngestionSourceDto` documentation in `ingestion.md` to reflect the current set of supported providers.

* updating tag

* feat: add REDIS_USER env variable (#172)

* feat: add REDIS_USER env variable

fixes #171

* add proper type for bullmq config

* Bulgarian UI language strings added (backend+frontend) (#194)

* Bulgarian UI Support added

* BG language UI support - Create translation.json

* update redis config logic

* Update Bulgarian language setting, register language

* Allow specifying local file path for mbox/eml/pst (#214)

* Add agents AI doc

* Allow local file path for Mbox file ingestion


---------

Co-authored-by: Wei S. <5291640+wayneshn@users.noreply.github.com>

* feat(ingestion): add local file path support and optimize EML processing

- Frontend: Updated IngestionSourceForm to allow toggling between "Upload File" and "Local File Path" for PST, EML, and Mbox providers.
- Frontend: Added logic to clear irrelevant form data when switching import methods.
- Frontend: Added English translations for new form fields.
- Backend: Refactored EMLConnector to stream ZIP entries using yauzl instead of extracting the full archive to disk, significantly improving efficiency for large archives.
- Docs: Updated API documentation and User Guides (PST, EML, Mbox) to clarify "Local File Path" usage, specifically within Docker environments.

* docs: add meilisearch dumpless upgrade guide and snapshot config

Update `docker-compose.yml` to include the `MEILI_SCHEDULE_SNAPSHOT` environment variable, defaulting to 86400 seconds (24 hours), enabling periodic data snapshots for easier recovery. Shout out to @morph027 for the inspiration!

Additionally, update the Meilisearch upgrade documentation to include an experimental "dumpless" upgrade guide while marking the previous method as the standard recommended process.

* build(coolify): enable daily snapshots for meilisearch

Configure the Meilisearch service in `open-archiver.yml` to create snapshots every 86400 seconds (24 hours) by setting the `MEILI_SCHEDULE_SNAPSHOT` environment variable.

---------

Co-authored-by: Antonia Schwennesen <53372671+zophiana@users.noreply.github.com>
Co-authored-by: IT Creativity + Art Team <admin@it-playground.net>
Co-authored-by: Jan Berdajs <mrbrdo@gmail.com>
2026-02-23 21:25:44 +01:00

5.7 KiB

Ingestion Service API

The Ingestion Service manages ingestion sources, which are configurations for connecting to email providers and importing emails.

Endpoints

All endpoints in this service require authentication.

POST /api/v1/ingestion-sources

Creates a new ingestion source.

Access: Authenticated

Request Body

The request body should be a CreateIngestionSourceDto object.

interface CreateIngestionSourceDto {
	name: string;
	provider: 'google_workspace' | 'microsoft_365' | 'generic_imap' | 'pst_import' | 'eml_import' | 'mbox_import';
	providerConfig: IngestionCredentials;
}

Example: Creating an Mbox Import Source with File Upload

{
	"name": "My Mbox Import",
	"provider": "mbox_import",
	"providerConfig": {
		"type": "mbox_import",
		"uploadedFileName": "emails.mbox",
		"uploadedFilePath": "open-archiver/tmp/uuid-emails.mbox"
	}
}

Example: Creating an Mbox Import Source with Local File Path

{
	"name": "My Mbox Import",
	"provider": "mbox_import",
	"providerConfig": {
		"type": "mbox_import",
		"localFilePath": "/path/to/emails.mbox"
	}
}

Note: When using localFilePath, the file will not be deleted after import. When using uploadedFilePath (via the upload API), the file will be automatically deleted after import. The same applies to pst_import and eml_import providers.

Important regarding localFilePath: When running OpenArchiver in a Docker container (which is the standard deployment), localFilePath refers to the path inside the Docker container, not on the host machine. To use a local file:

  1. Recommended: Place your file inside the directory defined by STORAGE_LOCAL_ROOT_PATH (e.g., inside a temp folder). Since this directory is already mounted as a volume, the file will be accessible at the same path inside the container.
  2. Alternative: Mount a specific directory containing your files as a volume in docker-compose.yml. For example, add - /path/to/my/files:/imports to the volumes section and use /imports/myfile.pst as the localFilePath.

Responses

  • 201 Created: The newly created ingestion source.
  • 500 Internal Server Error: An unexpected error occurred.

GET /api/v1/ingestion-sources

Retrieves all ingestion sources.

Access: Authenticated

Responses

  • 200 OK: An array of ingestion source objects.
  • 500 Internal Server Error: An unexpected error occurred.

GET /api/v1/ingestion-sources/:id

Retrieves a single ingestion source by its ID.

Access: Authenticated

URL Parameters

Parameter Type Description
id string The ID of the ingestion source.

Responses

  • 200 OK: The ingestion source object.
  • 404 Not Found: Ingestion source not found.
  • 500 Internal Server Error: An unexpected error occurred.

PUT /api/v1/ingestion-sources/:id

Updates an existing ingestion source.

Access: Authenticated

URL Parameters

Parameter Type Description
id string The ID of the ingestion source.

Request Body

The request body should be an UpdateIngestionSourceDto object.

interface UpdateIngestionSourceDto {
	name?: string;
	provider?: 'google' | 'microsoft' | 'generic_imap';
	providerConfig?: IngestionCredentials;
	status?: 'pending_auth' | 'auth_success' | 'importing' | 'active' | 'paused' | 'error';
}

Responses

  • 200 OK: The updated ingestion source object.
  • 404 Not Found: Ingestion source not found.
  • 500 Internal Server Error: An unexpected error occurred.

DELETE /api/v1/ingestion-sources/:id

Deletes an ingestion source and all associated data.

Access: Authenticated

URL Parameters

Parameter Type Description
id string The ID of the ingestion source.

Responses

  • 204 No Content: The ingestion source was deleted successfully.
  • 404 Not Found: Ingestion source not found.
  • 500 Internal Server Error: An unexpected error occurred.

POST /api/v1/ingestion-sources/:id/import

Triggers the initial import process for an ingestion source.

Access: Authenticated

URL Parameters

Parameter Type Description
id string The ID of the ingestion source.

Responses

  • 202 Accepted: The initial import was triggered successfully.
  • 404 Not Found: Ingestion source not found.
  • 500 Internal Server Error: An unexpected error occurred.

POST /api/v1/ingestion-sources/:id/pause

Pauses an active ingestion source.

Access: Authenticated

URL Parameters

Parameter Type Description
id string The ID of the ingestion source.

Responses

  • 200 OK: The updated ingestion source object with a paused status.
  • 404 Not Found: Ingestion source not found.
  • 500 Internal Server Error: An unexpected error occurred.

POST /api/v1/ingestion-sources/:id/sync

Triggers a forced synchronization for an ingestion source.

Access: Authenticated

URL Parameters

Parameter Type Description
id string The ID of the ingestion source.

Responses

  • 202 Accepted: The force sync was triggered successfully.
  • 404 Not Found: Ingestion source not found.
  • 500 Internal Server Error: An unexpected error occurred.