feat: improve S3-compatible OpenDAL configuration via URI query params #107

New Issue

MrUnknownDE · 2026-04-05T20:30:24+02:00

MrUnknownDE commented

2026-04-05 20:30:24 +02:00

Originally created by @g-roliveira on 2/17/2026

Summary

This PR improves S3-compatible object storage support by parsing and applying URI query parameters for OpenDAL S3 configuration across Vaultwarden storage paths.

It keeps AWS defaults unchanged while enabling robust provider-specific overrides for MinIO, Cloudflare R2, Ceph RGW, and similar S3-compatible endpoints.

What changed

Added centralized S3 URI parsing for paths like:
- DATA_FOLDER
- ATTACHMENTS_FOLDER
- ICON_CACHE_FOLDER
- SENDS_FOLDER
Added support for query parameters on s3://... URIs, including:
- endpoint
- region
- enable_virtual_host_style (including alias handling)
- default_storage_class (including empty value to omit storage class)
Enforced unknown parameter rejection in parsing logic.
Kept AWS-friendly defaults intact when no overrides are provided.

CI / tests

Added/updated unit tests for:
- defaults
- endpoint/path-style overrides
- storage class omission
- boolean variants and implicit flags
- percent-encoded path handling
- unknown parameter rejection
Added MinIO integration coverage in CI and stabilized startup/test execution.
Adjusted error assertion robustness for wrapped error formatting.

Docs

Updated .env.template with S3-compatible parameter guidance and examples.
Updated README.md with practical S3-compatible usage examples.
Added clarification that images must include both DB backend + s3 feature (sqlite,s3 / postgresql,s3 / mysql,s3).

Manual validation

Confirmed local MinIO Put/Get/Delete integration path.
Confirmed Cloudflare R2 Put/Get/Delete behavior using endpoint + region=auto + path-style.

Compatibility notes

AWS S3 behavior remains unchanged by default.
No migration required for existing non-S3 setups.
Existing S3 users without query params continue to work with current defaults.

Operational notes (browser downloads on S3-compatible storage)

When attachments are stored in S3-compatible backends, Web Vault downloads use presigned URLs and are fetched directly by the browser.

To make downloads work reliably, configure both sides:

Vaultwarden CSP: set ALLOWED_CONNECT_SRC to include the object-storage origin (for example https://<accountid>.r2.cloudflarestorage.com).
Object storage CORS policy: allow your Vaultwarden origin (DOMAIN) for GET/HEAD and return Access-Control-Allow-Origin.

Typical errors and causes:

violates the document's Content Security Policy -> missing/incorrect ALLOWED_CONNECT_SRC.
No 'Access-Control-Allow-Origin' header -> missing/incorrect bucket/provider CORS policy.

Follow-up suggestion

Potential future improvement: provide an optional server-side download mode/proxy to reduce CORS setup friction for some deployments.
Trade-offs: extra traffic through Vaultwarden, higher resource usage, and different scalability characteristics.

*Originally created by @g-roliveira on 2/17/2026* ## Summary This PR improves S3-compatible object storage support by parsing and applying URI query parameters for OpenDAL S3 configuration across Vaultwarden storage paths. It keeps AWS defaults unchanged while enabling robust provider-specific overrides for MinIO, Cloudflare R2, Ceph RGW, and similar S3-compatible endpoints. ## What changed - Added centralized S3 URI parsing for paths like: - `DATA_FOLDER` - `ATTACHMENTS_FOLDER` - `ICON_CACHE_FOLDER` - `SENDS_FOLDER` - Added support for query parameters on `s3://...` URIs, including: - `endpoint` - `region` - `enable_virtual_host_style` (including alias handling) - `default_storage_class` (including empty value to omit storage class) - Enforced unknown parameter rejection in parsing logic. - Kept AWS-friendly defaults intact when no overrides are provided. ## CI / tests - Added/updated unit tests for: - defaults - endpoint/path-style overrides - storage class omission - boolean variants and implicit flags - percent-encoded path handling - unknown parameter rejection - Added MinIO integration coverage in CI and stabilized startup/test execution. - Adjusted error assertion robustness for wrapped error formatting. ## Docs - Updated `.env.template` with S3-compatible parameter guidance and examples. - Updated `README.md` with practical S3-compatible usage examples. - Added clarification that images must include both DB backend + `s3` feature (`sqlite,s3` / `postgresql,s3` / `mysql,s3`). ## Manual validation - Confirmed local MinIO Put/Get/Delete integration path. - Confirmed Cloudflare R2 Put/Get/Delete behavior using endpoint + `region=auto` + path-style. ## Compatibility notes - AWS S3 behavior remains unchanged by default. - No migration required for existing non-S3 setups. - Existing S3 users without query params continue to work with current defaults. ## Operational notes (browser downloads on S3-compatible storage) When attachments are stored in S3-compatible backends, Web Vault downloads use presigned URLs and are fetched directly by the browser. To make downloads work reliably, configure both sides: - Vaultwarden CSP: set `ALLOWED_CONNECT_SRC` to include the object-storage origin (for example `https://<accountid>.r2.cloudflarestorage.com`). - Object storage CORS policy: allow your Vaultwarden origin (`DOMAIN`) for `GET`/`HEAD` and return `Access-Control-Allow-Origin`. Typical errors and causes: - `violates the document's Content Security Policy` -> missing/incorrect `ALLOWED_CONNECT_SRC`. - `No 'Access-Control-Allow-Origin' header` -> missing/incorrect bucket/provider CORS policy. ## Follow-up suggestion Potential future improvement: provide an optional server-side download mode/proxy to reduce CORS setup friction for some deployments. Trade-offs: extra traffic through Vaultwarden, higher resource usage, and different scalability characteristics.

Sign in to join this conversation.

No Label

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github/vaultwarden#107