* formatting code
* Remove uninstalled packages
* fix(imap): Improve IMAP connection stability and error handling
This commit refactors the IMAP connector to enhance connection management, error handling, and overall stability during email ingestion.
The `isConnected` flag has been removed in favor of relying directly on the `client.usable` property from the `imapflow` library. This simplifies the connection logic and avoids state synchronization issues.
The `connect` method now re-creates the client instance if it's not usable, ensuring a fresh connection after errors or disconnects. The retry mechanism (`withRetry`) has been updated to no longer manually reset the connection state, as the `connect` method now handles this automatically on the next attempt.
Additionally, a minor bug in the `sync-cycle-finished` processor has been fixed. The logic for merging sync states from successful jobs has been simplified and correctly typed, preventing potential runtime errors when no successful jobs are present.
---------
Co-authored-by: Wayne <5291640+ringoinca@users.noreply.github.com>
* Jobs page responsive fix
* feat(ingestion): Refactor email indexing into a dedicated background job
This commit refactors the email indexing process to improve the performance and reliability of the ingestion pipeline.
Previously, email indexing was performed synchronously within the mailbox processing job. This could lead to timeouts and failed ingestion cycles if the indexing step was slow or encountered errors.
To address this, the indexing logic has been moved into a separate, dedicated background job queue (`indexingQueue`). Now, the mailbox processor simply adds a batch of emails to this queue. A separate worker then processes the indexing job asynchronously.
This decoupling makes the ingestion process more robust:
- It prevents slow indexing from blocking or failing the entire mailbox sync.
- It allows for better resource management and scalability by handling indexing in a dedicated process.
- It improves error handling, as a failed indexing job can be retried independently without affecting the main ingestion flow.
Additionally, this commit includes minor documentation updates and removes a premature timeout in the PDF text extraction helper that was causing issues.
---------
Co-authored-by: Wayne <5291640+ringoinca@users.noreply.github.com>
This commit introduces two major features:
1. **Mbox File Ingestion:**
Users can now ingest emails from Mbox files (`.mbox`). A new Mbox connector has been implemented on the backend, and the user interface has been updated to support creating Mbox ingestion sources. Documentation for this new provider has also been added.
Additionally, this commit includes new documentation for upgrading and migrating Open Archiver.
Co-authored-by: Wayne <5291640+ringoinca@users.noreply.github.com>
