V0.5.1 dev (#341)

* OpenAPI root url fix

* Journaling OSS setup

* feat: add preserve-original-file mode for email ingestion for GoBD compliance

- Add `preserveOriginalFile` option to ingestion sources and connectors
- Stream original EML/MBOX/PST emails to temp files instead of holding
  full buffers in memory, reducing memory allocation during ingestion
- Skip attachment binary extraction and EML re-serialization when
  preserve mode is enabled; use raw file on disk as source of truth
- Update `EmailObject` to use `tempFilePath` instead of in-memory `eml`
  buffer across all connectors (EML, MBOX, PST)
- Add new database migration (0032) for `preserve_original_file` column
- Add frontend UI toggle with tooltip (tippy.js) for the new option
- Replace console.warn calls with structured pino logger in connectors

* add isjournaled property to archived_email

* feat(ingestion): add unmerge ingestion source functionality

Introduces the ability to detach a child ingestion source from its
merge group, making it a standalone root source. Changes include:

- Add `unmerge` controller method with auth and error handling
- Add POST `/v1/ingestion-sources/{id}/unmerge` route with OpenAPI docs
- Implement `IngestionService.unmerge` backend logic
- Add unmerge UI action and handler in the frontend ingestion view
- Fix bulk delete to also remove children of deleted root sources
- Update docs with new API operation and merging sources user guide

* code formatting

* Database migration file for enum `partially_active`

* Error handling improvement
This commit is contained in:
Wei S.
2026-03-30 22:29:03 +02:00
committed by GitHub
parent e5e119528f
commit 0c42b30c9e
76 changed files with 13130 additions and 531 deletions

View File

@@ -104,3 +104,24 @@ ENCRYPTION_KEY=
# Apache Tika Integration
# ONLY active if TIKA_URL is set
TIKA_URL=http://tika:9998
# Enterprise features (Skip this part if you are using the open-source version)
# Batch size for managing retention policy lifecycle. (This number of emails will be checked each time when retention policy scans the database. Adjust based on your system capability.)
RETENTION_BATCH_SIZE=1000
# --- SMTP Journaling (Enterprise only) ---
# The port the embedded SMTP journaling listener binds to inside the container.
# This is the port your MTA (Exchange, MS365, Postfix, etc.) will send journal reports to.
# The docker-compose.yml maps this same port on the host side by default.
SMTP_JOURNALING_PORT=2525
# The domain used to generate routing addresses for journaling sources.
# Each source gets a unique address like journal-<id>@<domain>.
# Set this to the domain/subdomain whose MX record points to this server.
SMTP_JOURNALING_DOMAIN=journal.yourdomain.com
# Maximum number of waiting jobs in the journal queue before the SMTP listener
# returns 4xx temporary failures (backpressure). The MTA will retry automatically.
JOURNAL_QUEUE_BACKPRESSURE_THRESHOLD=10000
#BullMQ worker concurrency for processing journaled emails. Increase on servers with more CPU cores.
JOURNAL_WORKER_CONCURRENCY=3