Compare commits

...

84 Commits

Author SHA1 Message Date
Rostislav Dugin
eb8e5aa428 FEATURE (storages): Add SFTP 2025-12-19 23:24:16 +03:00
github-actions[bot]
1f030bd8fb Update CITATION.cff to v2.8.1 2025-12-19 11:44:37 +00:00
Rostislav Dugin
b278a79104 FIX (databases): Remove optional text from db name field 2025-12-19 14:28:54 +03:00
github-actions[bot]
b74ae734af Update CITATION.cff to v2.8.0 2025-12-18 16:13:17 +00:00
Rostislav Dugin
d21a9398c6 FIX (Dockerfile): Upgrade Go version 2025-12-18 18:57:26 +03:00
Rostislav Dugin
6ad7b95b7d FIX (go tidy): Run go mod tidy 2025-12-18 18:42:02 +03:00
Rostislav Dugin
8432d1626f FIX (linting): Increase lint timeout 2025-12-18 18:36:11 +03:00
Rostislav Dugin
d7f631fa93 FIX (golangci): Upgrade version of golangci 2025-12-18 18:33:41 +03:00
Rostislav Dugin
c3fb2aa529 FIX (golangci): Upgrade version of golangci 2025-12-18 18:31:03 +03:00
Rostislav Dugin
1817937409 FIX (ci \ cd): Upgrade Go version 2025-12-18 18:16:37 +03:00
Rostislav Dugin
3172396668 FIX (extensions): Exclude extensions comments as well 2025-12-18 17:54:52 +03:00
Rostislav Dugin
9cd5c8c57c Merge branch 'main' of https://github.com/RostislavDugin/postgresus 2025-12-18 17:49:24 +03:00
Rostislav Dugin
d8826d85c3 FEATURE (storanges): Add rclone 2025-12-18 17:46:16 +03:00
github-actions[bot]
49fdd46cbe Update CITATION.cff to v2.7.0 2025-12-18 11:49:21 +00:00
Rostislav Dugin
c6261d434b FEATURE (restores): Allow to exclude extensions over restore 2025-12-18 14:34:32 +03:00
github-actions[bot]
918002acde Update CITATION.cff to v2.6.0 2025-12-17 14:03:33 +00:00
Rostislav Dugin
c0721a43e1 FEATURE (docs): Add code of conduct 2025-12-17 16:41:07 +03:00
Rostislav Dugin
461e15cd7a FEATURE (security): Add security md file 2025-12-17 16:33:10 +03:00
Rostislav Dugin
69a53936f5 FEATURE (citation): Add CITATION.cff 2025-12-17 16:17:43 +03:00
Rostislav Dugin
2bafec3c19 FIX (databases): Fix second opening of storage & notifier creation dialogs 2025-12-16 13:33:56 +03:00
Rostislav Dugin
422b44dfdc FEATURE (ftp): Get rid of passive mode 2025-12-14 00:01:21 +03:00
Rostislav Dugin
51d7fe54d0 Merge pull request #144 from omerkarabacak/main
FEATURE (clusters): Add cluster-based database management and bulk import
2025-12-13 22:37:35 +03:00
Omer Karabacak
6e2d63626c FEATURE (clusters): Add cluster-based database management and bulk import functionality 2025-12-13 20:32:54 +01:00
Rostislav Dugin
260c7a1188 FEATURE (frontend): Add frontend tests 2025-12-13 22:22:31 +03:00
Rostislav Dugin
ace94c144b FEATURE (storanges): Add FTP storange 2025-12-13 22:17:16 +03:00
Rostislav Dugin
b666cd9e2e Merge pull request #143 from RostislavDugin/develop
FEATURE (parsing): Add parsing connection string on DB creation
2025-12-13 13:53:30 +03:00
Rostislav Dugin
9dac63430d FEATURE (parsing): Add parsing connection string on DB creation 2025-12-13 13:50:22 +03:00
Rostislav Dugin
8217906c7a Merge pull request #139 from RostislavDugin/develop
Merge develop into main
2025-12-11 20:02:32 +03:00
Rostislav Dugin
db71a5ef7b FIX (databases): Add support dashed databases for read only users creation 2025-12-11 19:57:49 +03:00
Rostislav Dugin
df78e296b3 FEATURE (s3): Allow to skip TLS verification 2025-12-11 19:50:59 +03:00
Rostislav Dugin
fda3bf9b98 FEATURE (supabase): Add support of Supabase, schemas excluding and get rid of version in UI 2025-12-11 19:27:45 +03:00
pv-create
e19f449c60 FIX (readme): Fix typos and links
* fix typos

* fix link

* fix email param

---------

Co-authored-by: pavelvilkov <vilkovpy@mi-broker.ru>
2025-12-10 19:44:49 +03:00
Leonardo Flores
5944d7c4b6 feat(postgresus): Add schema filter for pg_dump and pg_restore (#131)
Add optional "Schemas" field to PostgreSQL database settings allowing
users to specify which schemas to include in backups (comma-separated).

This solves permission issues when backing up some of databases that
have restricted internal schemas (auth, storage, realtime).

Changes:
- Add schemas column to postgresql_databases table (migration)
- Update PostgresqlDatabase model with Schemas field
- Modify buildPgDumpArgs() to append --schema flags for each schema
- Modify pg_restore args to support --schema filtering on restore
- Add Schemas input field to frontend edit form with tooltip
- Display schemas in read-only database view

Example usage: Setting schemas to "public,drizzle" generates:
  pg_dump ... --schema public --schema drizzle
  pg_restore ... --schema public --schema drizzle
2025-12-10 13:19:15 +03:00
Unicorn-Zombie-Apocalypse
1f5c9d3d01 feat: Add support for custom Root CA configuration in Helm chart (#129)
* feat: Add support for custom Root CA configuration in Helm chart

* fix: Remove default value for customRootCA in Helm chart
2025-12-09 19:36:52 +03:00
Rostislav Dugin
d27b885fc1 FIX (postgresql): Fix version detection without minor version after major 2025-12-09 10:36:07 +03:00
Rostislav Dugin
45054bc4b5 FIX (readme): Update README about PITR 2025-12-08 22:20:41 +03:00
Rostislav Dugin
09f27019e8 FIX (postgresql): Use UTF-8 encoding for DB connection by default 2025-12-08 17:40:37 +03:00
Rostislav Dugin
cba8fdf49c FEATURE (core)!: Release 2.0 2025-12-08 10:41:36 +03:00
Rostislav Dugin
41c72cf7b6 FIX (buffering): Simplify buffering logic for localstorage 2025-12-07 19:40:40 +03:00
Rostislav Dugin
f04a8b7a82 FIX (backup): Add double buffering for local storange 2025-12-07 19:02:44 +03:00
Rostislav Dugin
552167e4ef FIX (logos): Update logos 2025-12-07 18:46:39 +03:00
Rostislav Dugin
be42cfab1f Merge branch 'main' of https://github.com/RostislavDugin/postgresus 2025-12-07 17:50:05 +03:00
Rostislav Dugin
ea34ced676 Merge pull request #124 from akalitenya/helm-values-tag-fix
Set default helm chart image tag to null
2025-12-07 17:49:21 +03:00
Rostislav Dugin
09cb1488b3 FIX (notifications): Get rid of password validation for email 2025-12-07 17:48:11 +03:00
Rostislav Dugin
b6518ef667 FIX (buffers): Increase copy buffer size 2025-12-07 17:44:35 +03:00
akalitenya
25c58e6209 set default image tag to null 2025-12-07 10:34:18 +05:00
Rostislav Dugin
97ee4b55c2 FIX (helm): Use standard namespace behavior instead of hardcoded values 2025-12-04 19:59:19 +03:00
Rostislav Dugin
12eea72392 FEATURE (helm): Use ClusterIP by default and add deployment to ghcr.io 2025-12-04 15:11:09 +03:00
Rostislav Dugin
75c88bac50 FIX (webhook): Escape webhook characters 2025-12-04 14:28:49 +03:00
Rostislav Dugin
ff1b6536bf FIX (connection): Add standard_conforming_strings param when building string to connect to PG 2025-12-03 18:42:49 +03:00
Rostislav Dugin
06197f986d FIX (chunking): Add backuping chunk by chunk without buffering in RAM and improve cancelation process 2025-12-03 17:35:43 +03:00
Rostislav Dugin
fe72e9e0a6 FIX (healthcheck): Clean up healthcheck interval receving when tab changed 2025-12-03 08:08:49 +03:00
Rostislav Dugin
640cceadbd FIX (docs): Extend docs with HTTP route support 2025-12-03 07:43:00 +03:00
Rostislav Dugin
80e573fcb3 Merge pull request #121 from tylerobara/feature/add_httproute_support
FEATURE helm: Adding support for HTTPRoutes
2025-12-03 07:34:20 +03:00
Tyler Obara
35498d83f1 adding support for httperoutes 2025-12-02 17:01:38 -05:00
Rostislav Dugin
77ae8d1ac7 FIX (helm): Fix Helm path in readmes 2025-12-02 17:43:43 +03:00
Rostislav Dugin
2f20845b3d Merge branch 'main' of https://github.com/RostislavDugin/postgresus 2025-12-02 17:41:02 +03:00
Rostislav Dugin
a3d3df4093 FIX (zoom): Disable zoom on iOS 2025-12-02 17:40:43 +03:00
Rostislav Dugin
8db83d40d5 FIX (mobile): Do not preselect card on mobile for DBs, notifiers and storanges 2025-12-02 17:37:03 +03:00
Rostislav Dugin
065ded37bd Merge pull request #119 from tylerobara/fix/helm_liveness_readiness
FIX Helm: Templates, Liveness and Readiness probes
2025-12-02 17:15:50 +03:00
Tyler Obara
71e801debb change helm dir 2025-12-02 08:44:46 -05:00
Tyler Obara
ffd4e3a27b fixing liveness and readiness probes 2025-12-02 08:02:26 -05:00
Rostislav Dugin
d2a9085591 FIX (dump): Get rid of extra encoding param when backup and restore 2025-12-02 12:54:07 +03:00
Rostislav Dugin
6f0152b60c FIX (helm): Get rid of ingress by default 2025-12-02 10:03:47 +03:00
Rostislav Dugin
7007236f2f FIX (email): Recrate client in case of auth error 2025-12-02 09:43:49 +03:00
Rostislav Dugin
db55cad310 Merge pull request #116 from RostislavDugin/feature/helm_chart
FIX (helm): Add git clone step
2025-12-02 00:02:13 +03:00
Rostislav Dugin
25bd096c81 FIX (helm): Add git clone step 2025-12-01 23:57:05 +03:00
Rostislav Dugin
7e98dd578c Merge pull request #115 from RostislavDugin/feature/helm_chart
Feature/helm chart
2025-12-01 23:47:27 +03:00
Rostislav Dugin
ba37b30e83 FEATURE (helm): Add Helm chart installation 2025-12-01 23:47:00 +03:00
Rostislav Dugin
34b3f822e3 Merge pull request #114 from spa-skyson/helmchart
helmchart v1.0.0
2025-12-01 23:18:20 +03:00
Rostislav Dugin
14700130b7 FIX (email): Add login auth in case if plain fails 2025-12-01 23:16:54 +03:00
Alexander Gazal
de11ab8d8a helmchart v1.0.0 2025-12-01 08:47:17 +03:00
Rostislav Dugin
06282bb435 FIX (connection): Avoid usage of prepare statements to get rid of problem with PgBounder 2025-11-30 20:50:25 +03:00
Rostislav Dugin
a3b263bbac FIX (installation): Fix installation on Debian 2025-11-30 20:25:28 +03:00
Rostislav Dugin
a956dccf7c FIX (whitelist): Show hint about Postgresus whitelist in case of connection failure 2025-11-28 23:59:20 +03:00
Rostislav Dugin
ce9fa18d58 FEATURE (webhook): Add webhook customization 2025-11-28 21:53:44 +03:00
Rostislav Dugin
281e185f21 FIX (dark): Add dark theme image 2025-11-27 23:17:43 +03:00
Rostislav Dugin
bb5b0064ea Merge branch 'main' of https://github.com/RostislavDugin/postgresus 2025-11-27 22:19:34 +03:00
Rostislav Dugin
da95bbb178 FIX (s3): Do not allow to change prefix after creation 2025-11-27 22:00:21 +03:00
Rostislav Dugin
cfe5993831 Merge pull request #110 from RostislavDugin/feature/pgpass_escape
Feature/pgpass escape
2025-11-27 17:03:06 +03:00
Rostislav Dugin
fa0e3d1ce2 REFACTOR (pgpass): Refactor escaping 2025-11-27 17:00:26 +03:00
Rostislav Dugin
d07085c462 Merge pull request #108 from kapawit/fix/pgpass-special-characters
FIX (postgresql): Escape special characters in .pgpass file for authentication
2025-11-27 16:54:38 +03:00
kapawit
c89c1f9654 FIX (postgresql): Escape special characters in .pgpass file for authentication 2025-11-26 21:35:38 +07:00
Rostislav Dugin
6cfc0ca79b FEATURE (dark): Add dark theme 2025-11-26 00:07:23 +03:00
155 changed files with 11090 additions and 1601 deletions

102
.github/CODE_OF_CONDUCT.md vendored Normal file
View File

@@ -0,0 +1,102 @@
# Code of Conduct
## Our Pledge
We as members, contributors and maintainers pledge to make participation in the Postgresus community a friendly and welcoming experience for everyone, regardless of background, experience level or personal circumstances.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive and healthy community.
## Our Standards
### Examples of behavior that contributes to a positive environment
- Using welcoming and inclusive language
- Being respectful of differing viewpoints and experiences
- Gracefully accepting constructive criticism
- Focusing on what is best for the community
- Showing empathy towards other community members
- Helping newcomers get started with contributions
- Providing clear and constructive feedback on pull requests
- Celebrating successes and acknowledging contributions
### Examples of unacceptable behavior
- Trolling, insulting or derogatory comments, and personal or political attacks
- Publishing others' private information, such as physical or email addresses, without their explicit permission
- Spam, self-promotion or off-topic content in project spaces
- Other conduct which could reasonably be considered inappropriate in a professional setting
## Scope
This Code of Conduct applies within all community spaces, including:
- GitHub repositories (issues, pull requests, discussions, comments)
- Telegram channels and direct messages related to Postgresus
- Social media interactions when representing the project
- Community forums and online discussions
- Any other spaces where Postgresus community members interact
This Code of Conduct also applies when an individual is officially representing the community in public spaces, such as using an official email address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
## Enforcement
Instances of abusive or unacceptable behavior may be reported to the community leaders responsible for enforcement:
- **Email**: [info@postgresus.com](mailto:info@postgresus.com)
- **Telegram**: [@rostislav_dugin](https://t.me/rostislav_dugin)
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of actions.
**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the community.
## Contributing with Respect
When contributing to Postgresus, please:
- Be patient with maintainers and other contributors
- Understand that everyone has different levels of experience
- Ask questions in a respectful manner
- Accept that your contribution may not be accepted, and be open to feedback
- Follow the [contribution guidelines](https://postgresus.com/contribute)
For code contributions, remember to:
- Discuss significant changes before implementing them
- Be open to code review feedback
- Help review others' contributions when possible
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html](https://www.contributor-covenant.org/version/2/1/code_of_conduct.html).
Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder](https://github.com/mozilla/diversity).
For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq](https://www.contributor-covenant.org/faq).

54
.github/SECURITY.md vendored Normal file
View File

@@ -0,0 +1,54 @@
# Security Policy
## Reporting a Vulnerability
If you discover a security vulnerability in Postgresus, please report it responsibly. **Do not create a public GitHub issue for security vulnerabilities.**
### How to Report
1. **Email** (preferred): Send details to [info@postgresus.com](mailto:info@postgresus.com)
2. **Telegram**: Contact [@rostislav_dugin](https://t.me/rostislav_dugin)
3. **GitHub Security Advisories**: Use the [private vulnerability reporting](https://github.com/RostislavDugin/postgresus/security/advisories/new) feature
### What to Include
- Description of the vulnerability
- Steps to reproduce the issue
- Potential impact and severity assessment
- Any suggested fixes (optional)
## Supported Versions
| Version | Supported |
| ------- | --------- |
| Latest | Yes |
We recommend always using the latest version of Postgresus. Security patches are applied to the most recent release.
### PostgreSQL Compatibility
Postgresus supports PostgreSQL versions 12, 13, 14, 15, 16, 17 and 18.
## Response Timeline
- **Acknowledgment**: Within 48-72 hours
- **Initial Assessment**: Within 1 week
- **Fix Timeline**: Depends on severity, but we aim to address critical issues as quickly as possible
We follow a coordinated disclosure policy. We ask that you give us reasonable time to address the vulnerability before any public disclosure.
## Security Features
Postgresus is designed with security in mind. For full details, see our [security documentation](https://postgresus.com/security).
Key features include:
- **AES-256-GCM Encryption**: Enterprise-grade encryption for backup files and sensitive data
- **Read-Only Database Access**: Postgresus uses read-only access by default and warns if write permissions are detected
- **Role-Based Access Control**: Assign viewer, member, admin or owner roles within workspaces
- **Audit Logging**: Track all system activities and changes made by users
- **Zero-Trust Storage**: Encrypted backups are safe even in shared cloud storage
## License
Postgresus is licensed under [Apache 2.0](../LICENSE).

View File

@@ -17,7 +17,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "1.23.3"
go-version: "1.24.4"
- name: Cache Go modules
uses: actions/cache@v4
@@ -31,7 +31,7 @@ jobs:
- name: Install golangci-lint
run: |
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.60.3
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/HEAD/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.7.2
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
- name: Install swag for swagger generation
@@ -82,6 +82,30 @@ jobs:
cd frontend
npm run lint
test-frontend:
runs-on: ubuntu-latest
needs: [lint-frontend]
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install dependencies
run: |
cd frontend
npm ci
- name: Run frontend tests
run: |
cd frontend
npm run test
test-backend:
runs-on: ubuntu-latest
needs: [lint-backend]
@@ -92,7 +116,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "1.23.3"
go-version: "1.24.4"
- name: Cache Go modules
uses: actions/cache@v4
@@ -141,9 +165,19 @@ jobs:
TEST_AZURITE_BLOB_PORT=10000
# testing NAS
TEST_NAS_PORT=7006
# testing FTP
TEST_FTP_PORT=7007
# testing SFTP
TEST_SFTP_PORT=7008
# testing Telegram
TEST_TELEGRAM_BOT_TOKEN=${{ secrets.TEST_TELEGRAM_BOT_TOKEN }}
TEST_TELEGRAM_CHAT_ID=${{ secrets.TEST_TELEGRAM_CHAT_ID }}
# supabase
TEST_SUPABASE_HOST=${{ secrets.TEST_SUPABASE_HOST }}
TEST_SUPABASE_PORT=${{ secrets.TEST_SUPABASE_PORT }}
TEST_SUPABASE_USERNAME=${{ secrets.TEST_SUPABASE_USERNAME }}
TEST_SUPABASE_PASSWORD=${{ secrets.TEST_SUPABASE_PASSWORD }}
TEST_SUPABASE_DATABASE=${{ secrets.TEST_SUPABASE_DATABASE }}
EOF
- name: Start test containers
@@ -170,6 +204,12 @@ jobs:
# Wait for Azurite
timeout 60 bash -c 'until nc -z localhost 10000; do sleep 2; done'
# Wait for FTP
timeout 60 bash -c 'until nc -z localhost 7007; do sleep 2; done'
# Wait for SFTP
timeout 60 bash -c 'until nc -z localhost 7008; do sleep 2; done'
- name: Create data and temp directories
run: |
# Create directories that are used for backups and restore
@@ -202,7 +242,7 @@ jobs:
determine-version:
runs-on: ubuntu-latest
needs: [test-backend, lint-frontend]
needs: [test-backend, test-frontend]
if: ${{ github.ref == 'refs/heads/main' && !contains(github.event.head_commit.message, '[skip-release]') }}
outputs:
should_release: ${{ steps.version_bump.outputs.should_release }}
@@ -295,7 +335,7 @@ jobs:
build-only:
runs-on: ubuntu-latest
needs: [test-backend, lint-frontend]
needs: [test-backend, test-frontend]
if: ${{ github.ref == 'refs/heads/main' && contains(github.event.head_commit.message, '[skip-release]') }}
steps:
- name: Check out code
@@ -455,6 +495,17 @@ jobs:
echo EOF
} >> $GITHUB_OUTPUT
- name: Update CITATION.cff version
run: |
VERSION="${{ needs.determine-version.outputs.new_version }}"
sed -i "s/^version: .*/version: ${VERSION}/" CITATION.cff
sed -i "s/^date-released: .*/date-released: \"$(date +%Y-%m-%d)\"/" CITATION.cff
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add CITATION.cff
git commit -m "Update CITATION.cff to v${VERSION}" || true
git push || true
- name: Create GitHub Release
uses: actions/create-release@v1
env:
@@ -465,3 +516,37 @@ jobs:
body: ${{ steps.changelog.outputs.changelog }}
draft: false
prerelease: false
publish-helm-chart:
runs-on: ubuntu-latest
needs: [determine-version, build-and-push]
if: ${{ needs.determine-version.outputs.should_release == 'true' }}
permissions:
contents: read
packages: write
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Set up Helm
uses: azure/setup-helm@v4
with:
version: v3.14.0
- name: Log in to GHCR
run: echo "${{ secrets.GITHUB_TOKEN }}" | helm registry login ghcr.io -u ${{ github.actor }} --password-stdin
- name: Update Chart.yaml with release version
run: |
VERSION="${{ needs.determine-version.outputs.new_version }}"
sed -i "s/^version: .*/version: ${VERSION}/" deploy/helm/Chart.yaml
sed -i "s/^appVersion: .*/appVersion: \"v${VERSION}\"/" deploy/helm/Chart.yaml
cat deploy/helm/Chart.yaml
- name: Package Helm chart
run: helm package deploy/helm --destination .
- name: Push Helm chart to GHCR
run: |
VERSION="${{ needs.determine-version.outputs.new_version }}"
helm push postgresus-${VERSION}.tgz oci://ghcr.io/rostislavdugin/charts

5
.gitignore vendored
View File

@@ -5,4 +5,7 @@ pgdata/
docker-compose.yml
node_modules/
.idea
/articles
/articles
.DS_Store
/scripts

33
CITATION.cff Normal file
View File

@@ -0,0 +1,33 @@
cff-version: 1.2.0
title: Postgresus
message: "If you use this software, please cite it as below."
type: software
authors:
- family-names: Dugin
given-names: Rostislav
repository-code: https://github.com/RostislavDugin/postgresus
url: https://postgresus.com
abstract: "Free, open source and self-hosted solution for automated PostgreSQL backups with multiple storage options and notifications."
keywords:
- docker
- kubernetes
- golang
- backups
- postgres
- devops
- backup
- database
- tools
- monitoring
- ftp
- postgresql
- s3
- psql
- web-ui
- self-hosted
- pg
- system-administration
- database-backup
license: Apache-2.0
version: 2.8.1
date-released: "2025-12-19"

View File

@@ -22,7 +22,7 @@ RUN npm run build
# ========= BUILD BACKEND =========
# Backend build stage
FROM --platform=$BUILDPLATFORM golang:1.23.3 AS backend-build
FROM --platform=$BUILDPLATFORM golang:1.24.4 AS backend-build
# Make TARGET args available early so tools built here match the final image arch
ARG TARGETOS
@@ -77,16 +77,16 @@ ENV APP_VERSION=$APP_VERSION
# Set production mode for Docker containers
ENV ENV_MODE=production
# Install PostgreSQL server and client tools (versions 12-18)
# Install PostgreSQL server and client tools (versions 12-18) and rclone
RUN apt-get update && apt-get install -y --no-install-recommends \
wget ca-certificates gnupg lsb-release sudo gosu && \
wget ca-certificates gnupg lsb-release sudo gosu curl unzip && \
wget -qO- https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - && \
echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" \
> /etc/apt/sources.list.d/pgdg.list && \
apt-get update && \
apt-get install -y --no-install-recommends \
postgresql-17 postgresql-18 postgresql-client-12 postgresql-client-13 postgresql-client-14 postgresql-client-15 \
postgresql-client-16 postgresql-client-17 postgresql-client-18 && \
postgresql-client-16 postgresql-client-17 postgresql-client-18 rclone && \
rm -rf /var/lib/apt/lists/*
# Create postgres user and set up directories

View File

@@ -25,6 +25,8 @@
<a href="https://postgresus.com" target="_blank"><strong>🌐 Postgresus website</strong></a>
</p>
<img src="assets/dashboard-dark.svg" alt="Postgresus Dark Dashboard" width="800" style="margin-bottom: 10px;"/>
<img src="assets/dashboard.svg" alt="Postgresus Dashboard" width="800"/>
@@ -72,6 +74,21 @@
- **Audit logs**: Track all system activities and changes made by users
- **User roles**: Assign viewer, member, admin or owner roles within workspaces
### 🎨 **UX-Friendly**
- **Designer-polished UI**: Clean, intuitive interface crafted with attention to detail
- **Dark & light themes**: Choose the look that suits your workflow
- **Mobile adaptive**: Check your backups from anywhere on any device
### ☁️ **Works with Self-Hosted & Cloud Databases**
Postgresus works seamlessly with both self-hosted PostgreSQL and cloud-managed databases:
- **Cloud support**: AWS RDS, Google Cloud SQL, Azure Database for PostgreSQL
- **Self-hosted**: Any PostgreSQL instance you manage yourself
- **Why no PITR?**: Cloud providers already offer native PITR, and external PITR backups cannot be restored to managed cloud databases — making them impractical for cloud-hosted PostgreSQL
- **Practical granularity**: Hourly and daily backups are sufficient for 99% of projects without the operational complexity of WAL archiving
### 🐳 **Self-Hosted & Secure**
- **Docker-based**: Easy deployment and management
@@ -80,7 +97,7 @@
### 📦 Installation <a href="https://postgresus.com/installation">(docs)</a>
You have three ways to install Postgresus:
You have several ways to install Postgresus:
- Script (recommended)
- Simple Docker run
@@ -98,7 +115,7 @@ You have three ways to install Postgresus: automated script (recommended), simpl
The installation script will:
- ✅ Install Docker with Docker Compose(if not already installed)
- ✅ Install Docker with Docker Compose (if not already installed)
- ✅ Set up Postgresus
- ✅ Configure automatic startup on system reboot
@@ -149,6 +166,46 @@ Then run:
docker compose up -d
```
### Option 4: Kubernetes with Helm
For Kubernetes deployments, install directly from the OCI registry.
**With ClusterIP + port-forward (development/testing):**
```bash
helm install postgresus oci://ghcr.io/rostislavdugin/charts/postgresus \
-n postgresus --create-namespace
```
```bash
kubectl port-forward svc/postgresus-service 4005:4005 -n postgresus
# Access at http://localhost:4005
```
**With LoadBalancer (cloud environments):**
```bash
helm install postgresus oci://ghcr.io/rostislavdugin/charts/postgresus \
-n postgresus --create-namespace \
--set service.type=LoadBalancer
```
```bash
kubectl get svc postgresus-service -n postgresus
# Access at http://<EXTERNAL-IP>:4005
```
**With Ingress (domain-based access):**
```bash
helm install postgresus oci://ghcr.io/rostislavdugin/charts/postgresus \
-n postgresus --create-namespace \
--set ingress.enabled=true \
--set ingress.hosts[0].host=backup.example.com
```
For more options (NodePort, TLS, HTTPRoute for Gateway API), see the [Helm chart README](deploy/helm/README.md).
---
## 🚀 Usage
@@ -175,10 +232,10 @@ Replace `admin` with the actual email address of the user whose password you wan
## 📝 License
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details.
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details
---
## 🤝 Contributing
Contributions are welcome! Read <a href="https://postgresus.com/contributing">contributing guide</a> for more details, prioerities and rules are specified there. If you want to contribute, but don't know what and how - message me on Telegram [@rostislav_dugin](https://t.me/rostislav_dugin)
Contributions are welcome! Read <a href="https://postgresus.com/contribute">contributing guide</a> for more details, priorities and rules are specified there. If you want to contribute, but don't know what and how - message me on Telegram [@rostislav_dugin](https://t.me/rostislav_dugin)

764
assets/dashboard-dark.svg Normal file

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 766 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 913 KiB

After

Width:  |  Height:  |  Size: 771 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 13 KiB

View File

@@ -9,4 +9,4 @@ When applying changes, do not forget to refactor old code.
You can shortify, make more readable, improve code quality, etc.
Common logic can be extracted to functions, constants, files, etc.
After each large change with more than ~50-100 lines of code - always run `make lint` (from backend root folder).
After each large change with more than ~50-100 lines of code - always run `make lint` (from backend root folder) and, if you change frontend, run `npm run format` (from frontend root folder).

View File

@@ -33,4 +33,14 @@ TEST_NAS_PORT=7006
TEST_TELEGRAM_BOT_TOKEN=
TEST_TELEGRAM_CHAT_ID=
# testing Azure Blob Storage
TEST_AZURITE_BLOB_PORT=10000
TEST_AZURITE_BLOB_PORT=10000
# supabase
TEST_SUPABASE_HOST=
TEST_SUPABASE_PORT=
TEST_SUPABASE_USERNAME=
TEST_SUPABASE_PASSWORD=
TEST_SUPABASE_DATABASE=
# FTP
TEST_FTP_PORT=7007
# SFTP
TEST_SFTP_PORT=7008

View File

@@ -1,7 +1,7 @@
version: "2"
run:
timeout: 1m
timeout: 5m
tests: false
concurrency: 4

View File

@@ -132,3 +132,25 @@ services:
-s "backups;/shared;yes;no;no;testuser"
-p
container_name: test-nas
# Test FTP server
test-ftp:
image: stilliard/pure-ftpd:latest
ports:
- "${TEST_FTP_PORT:-21}:21"
- "30000-30009:30000-30009"
environment:
- PUBLICHOST=localhost
- FTP_USER_NAME=testuser
- FTP_USER_PASS=testpassword
- FTP_USER_HOME=/home/ftpusers/testuser
- FTP_PASSIVE_PORTS=30000:30009
container_name: test-ftp
# Test SFTP server
test-sftp:
image: atmoz/sftp:latest
ports:
- "${TEST_SFTP_PORT:-7008}:22"
command: testuser:testpassword:1001::upload
container_name: test-sftp

View File

@@ -1,6 +1,6 @@
module postgresus-backend
go 1.23.3
go 1.24.4
require (
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0
@@ -12,35 +12,195 @@ require (
github.com/google/uuid v1.6.0
github.com/ilyakaznacheev/cleanenv v1.5.0
github.com/jackc/pgx/v5 v5.7.5
github.com/jlaffaye/ftp v0.2.1-0.20240918233326-1b970516f5d3
github.com/jmoiron/sqlx v1.4.0
github.com/joho/godotenv v1.5.1
github.com/lib/pq v1.10.9
github.com/minio/minio-go/v7 v7.0.92
github.com/shirou/gopsutil/v4 v4.25.5
github.com/minio/minio-go/v7 v7.0.97
github.com/pkg/sftp v1.13.10
github.com/rclone/rclone v1.72.1
github.com/shirou/gopsutil/v4 v4.25.10
github.com/stretchr/testify v1.11.1
github.com/swaggo/files v1.0.1
github.com/swaggo/gin-swagger v1.6.0
github.com/swaggo/swag v1.16.4
golang.org/x/crypto v0.41.0
golang.org/x/time v0.12.0
golang.org/x/crypto v0.46.0
golang.org/x/time v0.14.0
gorm.io/driver/postgres v1.5.11
gorm.io/gorm v1.26.1
)
require github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect
require (
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect
github.com/Azure/azure-sdk-for-go/sdk/storage/azfile v1.5.3 // indirect
github.com/Azure/go-ntlmssp v0.0.2-0.20251110135918-10b7b7e7cd26 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 // indirect
github.com/Files-com/files-sdk-go/v3 v3.2.264 // indirect
github.com/IBM/go-sdk-core/v5 v5.21.0 // indirect
github.com/Max-Sum/base32768 v0.0.0-20230304063302-18e6ce5945fd // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/ProtonMail/bcrypt v0.0.0-20211005172633-e235017c1baf // indirect
github.com/ProtonMail/gluon v0.17.1-0.20230724134000-308be39be96e // indirect
github.com/ProtonMail/go-crypto v1.3.0 // indirect
github.com/ProtonMail/go-mime v0.0.0-20230322103455-7d82a3887f2f // indirect
github.com/ProtonMail/go-srp v0.0.7 // indirect
github.com/ProtonMail/gopenpgp/v2 v2.9.0 // indirect
github.com/PuerkitoBio/goquery v1.10.3 // indirect
github.com/a1ex3/zstd-seekable-format-go/pkg v0.10.0 // indirect
github.com/abbot/go-http-auth v0.4.0 // indirect
github.com/anchore/go-lzo v0.1.0 // indirect
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/appscode/go-querystring v0.0.0-20170504095604-0126cfb3f1dc // indirect
github.com/aws/aws-sdk-go-v2 v1.39.6 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.3 // indirect
github.com/aws/aws-sdk-go-v2/config v1.31.17 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.18.21 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 // indirect
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.4 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.13 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.13 // indirect
github.com/aws/aws-sdk-go-v2/service/s3 v1.90.0 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.30.1 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.39.1 // indirect
github.com/aws/smithy-go v1.23.2 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/boombuler/barcode v1.1.0 // indirect
github.com/bradenaw/juniper v0.15.3 // indirect
github.com/bradfitz/iter v0.0.0-20191230175014-e8f45d346db8 // indirect
github.com/buengese/sgzip v0.1.1 // indirect
github.com/buger/jsonparser v1.1.1 // indirect
github.com/calebcase/tmpfile v1.0.3 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/chilts/sid v0.0.0-20190607042430-660e94789ec9 // indirect
github.com/clipperhouse/stringish v0.1.1 // indirect
github.com/clipperhouse/uax29/v2 v2.3.0 // indirect
github.com/cloudflare/circl v1.6.1 // indirect
github.com/cloudinary/cloudinary-go/v2 v2.13.0 // indirect
github.com/cloudsoda/go-smb2 v0.0.0-20250228001242-d4c70e6251cc // indirect
github.com/cloudsoda/sddl v0.0.0-20250224235906-926454e91efc // indirect
github.com/colinmarc/hdfs/v2 v2.4.0 // indirect
github.com/coreos/go-semver v0.3.1 // indirect
github.com/coreos/go-systemd/v22 v22.6.0 // indirect
github.com/creasty/defaults v1.8.0 // indirect
github.com/cronokirby/saferith v0.33.0 // indirect
github.com/diskfs/go-diskfs v1.7.0 // indirect
github.com/dropbox/dropbox-sdk-go-unofficial/v6 v6.0.5 // indirect
github.com/emersion/go-message v0.18.2 // indirect
github.com/emersion/go-vcard v0.0.0-20241024213814-c9703dde27ff // indirect
github.com/flynn/noise v1.1.0 // indirect
github.com/go-chi/chi/v5 v5.2.3 // indirect
github.com/go-darwin/apfs v0.0.0-20211011131704-f84b94dbf348 // indirect
github.com/go-git/go-billy/v5 v5.6.2 // indirect
github.com/go-openapi/errors v0.22.4 // indirect
github.com/go-openapi/strfmt v0.25.0 // indirect
github.com/go-resty/resty/v2 v2.16.5 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/gofrs/flock v0.13.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v5 v5.3.0 // indirect
github.com/google/btree v1.1.3 // indirect
github.com/gorilla/schema v1.4.1 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-retryablehttp v0.7.8 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/henrybear327/Proton-API-Bridge v1.0.0 // indirect
github.com/henrybear327/go-proton-api v1.0.0 // indirect
github.com/jcmturner/aescts/v2 v2.0.0 // indirect
github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect
github.com/jcmturner/gofork v1.7.6 // indirect
github.com/jcmturner/goidentity/v6 v6.0.1 // indirect
github.com/jcmturner/gokrb5/v8 v8.4.4 // indirect
github.com/jcmturner/rpc/v2 v2.0.3 // indirect
github.com/jtolio/noiseconn v0.0.0-20231127013910-f6d9ecbf1de7 // indirect
github.com/jzelinskie/whirlpool v0.0.0-20201016144138-0675e54bb004 // indirect
github.com/klauspost/crc32 v1.3.0 // indirect
github.com/koofr/go-httpclient v0.0.0-20240520111329-e20f8f203988 // indirect
github.com/koofr/go-koofrclient v0.0.0-20221207135200-cbd7fc9ad6a6 // indirect
github.com/kr/fs v0.1.0 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/lanrat/extsort v1.4.2 // indirect
github.com/lpar/date v1.0.0 // indirect
github.com/lufia/plan9stats v0.0.0-20251013123823-9fd1530e3ec3 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-runewidth v0.0.19 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/ncw/swift/v2 v2.0.5 // indirect
github.com/oklog/ulid v1.3.1 // indirect
github.com/oracle/oci-go-sdk/v65 v65.104.0 // indirect
github.com/panjf2000/ants/v2 v2.11.3 // indirect
github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
github.com/pengsrc/go-shared v0.2.1-0.20190131101655-1999055a4a14 // indirect
github.com/peterh/liner v1.2.2 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pkg/xattr v0.4.12 // indirect
github.com/pquerna/otp v1.5.0 // indirect
github.com/prometheus/client_golang v1.23.2 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.67.2 // indirect
github.com/prometheus/procfs v0.19.2 // indirect
github.com/putdotio/go-putio/putio v0.0.0-20200123120452-16d982cac2b8 // indirect
github.com/relvacode/iso8601 v1.7.0 // indirect
github.com/rfjakob/eme v1.1.2 // indirect
github.com/sabhiram/go-gitignore v0.0.0-20210923224102-525f6e181f06 // indirect
github.com/samber/lo v1.52.0 // indirect
github.com/sirupsen/logrus v1.9.4-0.20230606125235-dd1b4c2e81af // indirect
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 // indirect
github.com/sony/gobreaker v1.0.0 // indirect
github.com/spacemonkeygo/monkit/v3 v3.0.25-0.20251022131615-eb24eb109368 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/t3rm1n4l/go-mega v0.0.0-20251031123324-a804aaa87491 // indirect
github.com/tklauser/go-sysconf v0.3.15 // indirect
github.com/tklauser/numcpus v0.10.0 // indirect
github.com/ulikunitz/xz v0.5.15 // indirect
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
github.com/yunify/qingstor-sdk-go/v3 v3.2.0 // indirect
github.com/zeebo/blake3 v0.2.4 // indirect
github.com/zeebo/errs v1.4.0 // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
go.etcd.io/bbolt v1.4.3 // indirect
go.mongodb.org/mongo-driver v1.17.6 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect
golang.org/x/term v0.38.0 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/validator.v2 v2.0.1 // indirect
moul.io/http2curl/v2 v2.3.0 // indirect
sigs.k8s.io/yaml v1.6.0 // indirect
storj.io/common v0.0.0-20251107171817-6221ae45072c // indirect
storj.io/drpc v0.0.35-0.20250513201419-f7819ea69b55 // indirect
storj.io/eventkit v0.0.0-20250410172343-61f26d3de156 // indirect
storj.io/infectious v0.0.2 // indirect
storj.io/picobuf v0.0.4 // indirect
storj.io/uplink v1.13.1 // indirect
)
require (
cloud.google.com/go/auth v0.16.2 // indirect
cloud.google.com/go/auth v0.17.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.7.0 // indirect
github.com/geoffgarside/ber v1.1.0 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
github.com/geoffgarside/ber v1.2.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
github.com/googleapis/gax-go/v2 v2.14.2 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.7 // indirect
github.com/googleapis/gax-go/v2 v2.15.0 // indirect
github.com/hirochachacha/go-smb2 v1.1.0
google.golang.org/genproto/googleapis/api v0.0.0-20250528174236-200df99c418a // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250603155806-513f23925822 // indirect
google.golang.org/grpc v1.73.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20251103181224-f26f9409b101 // indirect
google.golang.org/grpc v1.76.0 // indirect
)
require (
@@ -51,11 +211,11 @@ require (
github.com/bytedance/sonic v1.13.2 // indirect
github.com/bytedance/sonic/loader v0.2.4 // indirect
github.com/cloudwego/base64x v0.1.5 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/ebitengine/purego v0.8.4 // indirect
github.com/ebitengine/purego v0.9.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/gabriel-vasile/mimetype v1.4.9 // indirect
github.com/gabriel-vasile/mimetype v1.4.11 // indirect
github.com/gin-contrib/sse v1.1.0 // indirect
github.com/go-ini/ini v1.67.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
@@ -67,7 +227,7 @@ require (
github.com/go-openapi/swag v0.19.15 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.26.0 // indirect
github.com/go-playground/validator/v10 v10.28.0 // indirect
github.com/go-sql-driver/mysql v1.9.2 // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
@@ -77,40 +237,39 @@ require (
github.com/jinzhu/now v1.1.5 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.10 // indirect
github.com/klauspost/compress v1.18.1 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mailru/easyjson v0.7.6 // indirect
github.com/mailru/easyjson v0.9.1 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/minio/crc64nvme v1.0.1 // indirect
github.com/minio/crc64nvme v1.1.1 // indirect
github.com/minio/md5-simd v1.1.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/philhofer/fwd v1.2.0 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/rogpeppe/go-internal v1.14.1 // indirect
github.com/rs/xid v1.6.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/tinylib/msgp v1.3.0 // indirect
github.com/tinylib/msgp v1.5.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.12 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.36.0 // indirect
go.opentelemetry.io/otel/metric v1.36.0 // indirect
go.opentelemetry.io/otel/trace v1.36.0 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 // indirect
go.opentelemetry.io/otel v1.38.0 // indirect
go.opentelemetry.io/otel/metric v1.38.0 // indirect
go.opentelemetry.io/otel/trace v1.38.0 // indirect
golang.org/x/arch v0.17.0 // indirect
golang.org/x/net v0.43.0 // indirect
golang.org/x/oauth2 v0.30.0
golang.org/x/sync v0.16.0 // indirect
golang.org/x/sys v0.35.0 // indirect
golang.org/x/text v0.28.0 // indirect
golang.org/x/tools v0.35.0 // indirect
google.golang.org/api v0.239.0
google.golang.org/protobuf v1.36.6 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/oauth2 v0.33.0
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.39.0 // indirect
golang.org/x/text v0.32.0 // indirect
golang.org/x/tools v0.39.0 // indirect
google.golang.org/api v0.255.0
google.golang.org/protobuf v1.36.10 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
olympos.io/encoding/edn v0.0.0-20201019073823-d3554ca0b0a3 // indirect

File diff suppressed because it is too large Load Diff

View File

@@ -47,7 +47,9 @@ type EnvVariables struct {
TestAzuriteBlobPort string `env:"TEST_AZURITE_BLOB_PORT"`
TestNASPort string `env:"TEST_NAS_PORT"`
TestNASPort string `env:"TEST_NAS_PORT"`
TestFTPPort string `env:"TEST_FTP_PORT"`
TestSFTPPort string `env:"TEST_SFTP_PORT"`
// oauth
GitHubClientID string `env:"GITHUB_CLIENT_ID"`
@@ -58,6 +60,13 @@ type EnvVariables struct {
// testing Telegram
TestTelegramBotToken string `env:"TEST_TELEGRAM_BOT_TOKEN"`
TestTelegramChatID string `env:"TEST_TELEGRAM_CHAT_ID"`
// testing Supabase
TestSupabaseHost string `env:"TEST_SUPABASE_HOST"`
TestSupabasePort string `env:"TEST_SUPABASE_PORT"`
TestSupabaseUsername string `env:"TEST_SUPABASE_USERNAME"`
TestSupabasePassword string `env:"TEST_SUPABASE_PASSWORD"`
TestSupabaseDatabase string `env:"TEST_SUPABASE_DATABASE"`
}
var (

View File

@@ -2,20 +2,21 @@ package backups
import (
"context"
"errors"
"sync"
"github.com/google/uuid"
)
type BackupContextManager struct {
mu sync.RWMutex
cancelFuncs map[uuid.UUID]context.CancelFunc
mu sync.RWMutex
cancelFuncs map[uuid.UUID]context.CancelFunc
cancelledBackups map[uuid.UUID]bool
}
func NewBackupContextManager() *BackupContextManager {
return &BackupContextManager{
cancelFuncs: make(map[uuid.UUID]context.CancelFunc),
cancelFuncs: make(map[uuid.UUID]context.CancelFunc),
cancelledBackups: make(map[uuid.UUID]bool),
}
}
@@ -23,25 +24,37 @@ func (m *BackupContextManager) RegisterBackup(backupID uuid.UUID, cancelFunc con
m.mu.Lock()
defer m.mu.Unlock()
m.cancelFuncs[backupID] = cancelFunc
delete(m.cancelledBackups, backupID)
}
func (m *BackupContextManager) CancelBackup(backupID uuid.UUID) error {
m.mu.Lock()
defer m.mu.Unlock()
cancelFunc, exists := m.cancelFuncs[backupID]
if !exists {
return errors.New("backup is not in progress or already completed")
if m.cancelledBackups[backupID] {
return nil
}
cancelFunc()
delete(m.cancelFuncs, backupID)
cancelFunc, exists := m.cancelFuncs[backupID]
if exists {
cancelFunc()
delete(m.cancelFuncs, backupID)
}
m.cancelledBackups[backupID] = true
return nil
}
func (m *BackupContextManager) IsCancelled(backupID uuid.UUID) bool {
m.mu.RLock()
defer m.mu.RUnlock()
return m.cancelledBackups[backupID]
}
func (m *BackupContextManager) UnregisterBackup(backupID uuid.UUID) {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.cancelFuncs, backupID)
delete(m.cancelledBackups, backupID)
}

View File

@@ -1,6 +1,7 @@
package backups
import (
"context"
"encoding/json"
"fmt"
"io"
@@ -701,7 +702,7 @@ func createTestBackup(
dummyContent := []byte("dummy backup content for testing")
reader := strings.NewReader(string(dummyContent))
logger := slog.New(slog.NewTextHandler(io.Discard, nil))
if err := storages[0].SaveFile(encryption.GetFieldEncryptor(), logger, backup.ID, reader); err != nil {
if err := storages[0].SaveFile(context.Background(), encryption.GetFieldEncryptor(), logger, backup.ID, reader); err != nil {
panic(fmt.Sprintf("Failed to create test backup file: %v", err))
}

View File

@@ -16,7 +16,7 @@ const (
NonceLen = 12
ReservedLen = 12
HeaderLen = MagicBytesLen + SaltLen + NonceLen + ReservedLen
ChunkSize = 32 * 1024
ChunkSize = 1 * 1024 * 1024
PBKDF2Iterations = 100000
)

View File

@@ -275,7 +275,12 @@ func (s *BackupService) MakeBackup(databaseID uuid.UUID, isLastTry bool) {
errMsg := err.Error()
// Check if backup was cancelled (not due to shutdown)
if strings.Contains(errMsg, "backup cancelled") && !strings.Contains(errMsg, "shutdown") {
isCancelled := strings.Contains(errMsg, "backup cancelled") ||
strings.Contains(errMsg, "context canceled") ||
errors.Is(err, context.Canceled)
isShutdown := strings.Contains(errMsg, "shutdown")
if isCancelled && !isShutdown {
backup.Status = BackupStatusCanceled
backup.BackupDurationMs = time.Since(start).Milliseconds()
backup.BackupSizeMb = 0

View File

@@ -30,7 +30,7 @@ import (
const (
backupTimeout = 23 * time.Hour
shutdownCheckInterval = 1 * time.Second
copyBufferSize = 32 * 1024
copyBufferSize = 8 * 1024 * 1024
progressReportIntervalMB = 1.0
pgConnectTimeout = 30
compressionLevel = 5
@@ -45,6 +45,11 @@ type CreatePostgresqlBackupUsecase struct {
fieldEncryptor encryption.FieldEncryptor
}
type writeResult struct {
bytesWritten int
writeErr error
}
// Execute creates a backup of the database
func (uc *CreatePostgresqlBackupUsecase) Execute(
ctx context.Context,
@@ -126,7 +131,8 @@ func (uc *CreatePostgresqlBackupUsecase) streamToStorage(
}
defer func() {
if pgpassFile != "" {
_ = os.Remove(pgpassFile)
// Remove the entire temp directory (which contains the .pgpass file)
_ = os.RemoveAll(filepath.Dir(pgpassFile))
}
}()
@@ -172,7 +178,7 @@ func (uc *CreatePostgresqlBackupUsecase) streamToStorage(
// Start streaming into storage in its own goroutine
saveErrCh := make(chan error, 1)
go func() {
saveErr := storage.SaveFile(uc.fieldEncryptor, uc.logger, backupID, storageReader)
saveErr := storage.SaveFile(ctx, uc.fieldEncryptor, uc.logger, backupID, storageReader)
saveErrCh <- saveErr
}()
@@ -195,12 +201,10 @@ func (uc *CreatePostgresqlBackupUsecase) streamToStorage(
copyResultCh <- err
}()
// Wait for the copy to finish first, then the dump process
copyErr := <-copyResultCh
bytesWritten := <-bytesWrittenCh
waitErr := cmd.Wait()
// Check for shutdown or cancellation before finalizing
select {
case <-ctx.Done():
uc.cleanupOnCancellation(encryptionWriter, storageWriter, saveErrCh)
@@ -213,7 +217,6 @@ func (uc *CreatePostgresqlBackupUsecase) streamToStorage(
return nil, err
}
// Wait until storage ends reading
saveErr := <-saveErrCh
stderrOutput := <-stderrCh
@@ -267,7 +270,23 @@ func (uc *CreatePostgresqlBackupUsecase) copyWithShutdownCheck(
bytesRead, readErr := src.Read(buf)
if bytesRead > 0 {
bytesWritten, writeErr := dst.Write(buf[0:bytesRead])
writeResultCh := make(chan writeResult, 1)
go func() {
bytesWritten, writeErr := dst.Write(buf[0:bytesRead])
writeResultCh <- writeResult{bytesWritten, writeErr}
}()
var bytesWritten int
var writeErr error
select {
case <-ctx.Done():
return totalBytesWritten, fmt.Errorf("copy cancelled during write: %w", ctx.Err())
case result := <-writeResultCh:
bytesWritten = result.bytesWritten
writeErr = result.writeErr
}
if bytesWritten < 0 || bytesRead < bytesWritten {
bytesWritten = 0
if writeErr == nil {
@@ -316,6 +335,10 @@ func (uc *CreatePostgresqlBackupUsecase) buildPgDumpArgs(pg *pgtypes.PostgresqlD
"--verbose",
}
for _, schema := range pg.IncludeSchemas {
args = append(args, "-n", schema)
}
compressionArgs := uc.getCompressionArgs(pg.Version)
return append(args, compressionArgs...)
}
@@ -354,6 +377,9 @@ func (uc *CreatePostgresqlBackupUsecase) createBackupContext(
select {
case <-ctx.Done():
return
case <-parentCtx.Done():
cancel()
return
case <-ticker.C:
if config.IsShouldShutdown() {
cancel()
@@ -417,7 +443,6 @@ func (uc *CreatePostgresqlBackupUsecase) setupPgEnvironment(
"PGCONNECT_TIMEOUT="+strconv.Itoa(pgConnectTimeout),
"LC_ALL=C.UTF-8",
"LANG=C.UTF-8",
"PGOPTIONS=--client-encoding=UTF8",
)
if shouldRequireSSL {
@@ -611,7 +636,6 @@ func (uc *CreatePostgresqlBackupUsecase) handleExitCode1NoStderr(
"PGCONNECT_TIMEOUT=" + strconv.Itoa(pgConnectTimeout),
"LC_ALL=C.UTF-8",
"LANG=C.UTF-8",
"PGOPTIONS=--client-encoding=UTF8",
},
)
@@ -719,11 +743,15 @@ func (uc *CreatePostgresqlBackupUsecase) createTempPgpassFile(
return "", nil
}
escapedHost := tools.EscapePgpassField(pgConfig.Host)
escapedUsername := tools.EscapePgpassField(pgConfig.Username)
escapedPassword := tools.EscapePgpassField(password)
pgpassContent := fmt.Sprintf("%s:%d:*:%s:%s",
pgConfig.Host,
escapedHost,
pgConfig.Port,
pgConfig.Username,
password,
escapedUsername,
escapedPassword,
)
tempDir, err := os.MkdirTemp("", "pgpass")

View File

@@ -13,6 +13,7 @@ import (
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
"gorm.io/gorm"
)
type PostgresqlDatabase struct {
@@ -29,17 +30,40 @@ type PostgresqlDatabase struct {
Password string `json:"password" gorm:"type:text;not null"`
Database *string `json:"database" gorm:"type:text"`
IsHttps bool `json:"isHttps" gorm:"type:boolean;default:false"`
// backup settings
IncludeSchemas []string `json:"includeSchemas" gorm:"-"`
IncludeSchemasString string `json:"-" gorm:"column:include_schemas;type:text;not null;default:''"`
// restore settings (not saved to DB)
IsExcludeExtensions bool `json:"isExcludeExtensions" gorm:"-"`
}
func (p *PostgresqlDatabase) TableName() string {
return "postgresql_databases"
}
func (p *PostgresqlDatabase) Validate() error {
if p.Version == "" {
return errors.New("version is required")
func (p *PostgresqlDatabase) BeforeSave(_ *gorm.DB) error {
if len(p.IncludeSchemas) > 0 {
p.IncludeSchemasString = strings.Join(p.IncludeSchemas, ",")
} else {
p.IncludeSchemasString = ""
}
return nil
}
func (p *PostgresqlDatabase) AfterFind(_ *gorm.DB) error {
if p.IncludeSchemasString != "" {
p.IncludeSchemas = strings.Split(p.IncludeSchemasString, ",")
} else {
p.IncludeSchemas = []string{}
}
return nil
}
func (p *PostgresqlDatabase) Validate() error {
if p.Host == "" {
return errors.New("host is required")
}
@@ -85,6 +109,7 @@ func (p *PostgresqlDatabase) Update(incoming *PostgresqlDatabase) {
p.Username = incoming.Username
p.Database = incoming.Database
p.IsHttps = incoming.IsHttps
p.IncludeSchemas = incoming.IncludeSchemas
if incoming.Password != "" {
p.Password = incoming.Password
@@ -106,6 +131,50 @@ func (p *PostgresqlDatabase) EncryptSensitiveFields(
return nil
}
// PopulateVersionIfEmpty detects and sets the PostgreSQL version if not already set.
// This should be called before encrypting sensitive fields.
func (p *PostgresqlDatabase) PopulateVersionIfEmpty(
logger *slog.Logger,
encryptor encryption.FieldEncryptor,
databaseID uuid.UUID,
) error {
if p.Version != "" {
return nil
}
if p.Database == nil || *p.Database == "" {
return nil
}
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
password, err := decryptPasswordIfNeeded(p.Password, encryptor, databaseID)
if err != nil {
return fmt.Errorf("failed to decrypt password: %w", err)
}
connStr := buildConnectionStringForDB(p, *p.Database, password)
conn, err := pgx.Connect(ctx, connStr)
if err != nil {
return fmt.Errorf("failed to connect to database: %w", err)
}
defer func() {
if closeErr := conn.Close(ctx); closeErr != nil {
logger.Error("Failed to close connection", "error", closeErr)
}
}()
detectedVersion, err := detectDatabaseVersion(ctx, conn)
if err != nil {
return err
}
p.Version = detectedVersion
return nil
}
// IsUserReadOnly checks if the database user has read-only privileges.
//
// This method performs a comprehensive security check by examining:
@@ -286,8 +355,20 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
// Retry logic for username collision
maxRetries := 3
for attempt := 0; attempt < maxRetries; attempt++ {
username := fmt.Sprintf("postgresus-%s", uuid.New().String()[:8])
for attempt := range maxRetries {
// Generate base username for PostgreSQL user creation
baseUsername := fmt.Sprintf("postgresus-%s", uuid.New().String()[:8])
// For Supabase session pooler, the username format for connection is "username.projectid"
// but the actual PostgreSQL user must be created with just the base name.
// The pooler will strip the ".projectid" suffix when authenticating.
connectionUsername := baseUsername
if isSupabaseConnection(p.Host, p.Username) {
if supabaseProjectID := extractSupabaseProjectID(p.Username); supabaseProjectID != "" {
connectionUsername = fmt.Sprintf("%s.%s", baseUsername, supabaseProjectID)
}
}
newPassword := uuid.New().String()
tx, err := conn.Begin(ctx)
@@ -305,9 +386,10 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
}()
// Step 1: Create PostgreSQL user with LOGIN privilege
// Note: We use baseUsername for the actual PostgreSQL user name if Supabase is used
_, err = tx.Exec(
ctx,
fmt.Sprintf(`CREATE USER "%s" WITH PASSWORD '%s' LOGIN`, username, newPassword),
fmt.Sprintf(`CREATE USER "%s" WITH PASSWORD '%s' LOGIN`, baseUsername, newPassword),
)
if err != nil {
if err.Error() != "" && attempt < maxRetries-1 {
@@ -331,28 +413,28 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
}
// Now revoke from the specific user as well (belt and suspenders)
_, err = tx.Exec(ctx, fmt.Sprintf(`REVOKE CREATE ON SCHEMA public FROM "%s"`, username))
_, err = tx.Exec(ctx, fmt.Sprintf(`REVOKE CREATE ON SCHEMA public FROM "%s"`, baseUsername))
if err != nil {
logger.Error(
"Failed to revoke CREATE on public schema from user",
"error",
err,
"username",
username,
baseUsername,
)
}
// Step 2: Grant database connection privilege and revoke TEMP
_, err = tx.Exec(
ctx,
fmt.Sprintf(`GRANT CONNECT ON DATABASE %s TO "%s"`, *p.Database, username),
fmt.Sprintf(`GRANT CONNECT ON DATABASE "%s" TO "%s"`, *p.Database, baseUsername),
)
if err != nil {
return "", "", fmt.Errorf("failed to grant connect privilege: %w", err)
}
// Revoke TEMP privilege from PUBLIC role (like CREATE on public schema, TEMP is granted to PUBLIC by default)
_, err = tx.Exec(ctx, fmt.Sprintf(`REVOKE TEMP ON DATABASE %s FROM PUBLIC`, *p.Database))
_, err = tx.Exec(ctx, fmt.Sprintf(`REVOKE TEMP ON DATABASE "%s" FROM PUBLIC`, *p.Database))
if err != nil {
logger.Warn("Failed to revoke TEMP from PUBLIC", "error", err)
}
@@ -360,10 +442,10 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
// Also revoke from the specific user (belt and suspenders)
_, err = tx.Exec(
ctx,
fmt.Sprintf(`REVOKE TEMP ON DATABASE %s FROM "%s"`, *p.Database, username),
fmt.Sprintf(`REVOKE TEMP ON DATABASE "%s" FROM "%s"`, *p.Database, baseUsername),
)
if err != nil {
logger.Warn("Failed to revoke TEMP privilege", "error", err, "username", username)
logger.Warn("Failed to revoke TEMP privilege", "error", err, "username", baseUsername)
}
// Step 3: Discover all user-created schemas
@@ -396,7 +478,7 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
// Revoke CREATE specifically (handles inheritance from PUBLIC role)
_, err = tx.Exec(
ctx,
fmt.Sprintf(`REVOKE CREATE ON SCHEMA "%s" FROM "%s"`, schema, username),
fmt.Sprintf(`REVOKE CREATE ON SCHEMA "%s" FROM "%s"`, schema, baseUsername),
)
if err != nil {
logger.Warn(
@@ -406,14 +488,14 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
"schema",
schema,
"username",
username,
baseUsername,
)
}
// Grant only USAGE (not CREATE)
_, err = tx.Exec(
ctx,
fmt.Sprintf(`GRANT USAGE ON SCHEMA "%s" TO "%s"`, schema, username),
fmt.Sprintf(`GRANT USAGE ON SCHEMA "%s" TO "%s"`, schema, baseUsername),
)
if err != nil {
return "", "", fmt.Errorf("failed to grant usage on schema %s: %w", schema, err)
@@ -435,7 +517,7 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
EXECUTE format('GRANT SELECT ON ALL SEQUENCES IN SCHEMA %%I TO "%s"', schema_rec.schema_name);
END LOOP;
END $$;
`, username, username)
`, baseUsername, baseUsername)
_, err = tx.Exec(ctx, grantSelectSQL)
if err != nil {
@@ -457,7 +539,7 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
EXECUTE format('ALTER DEFAULT PRIVILEGES IN SCHEMA %%I GRANT SELECT ON SEQUENCES TO "%s"', schema_rec.schema_name);
END LOOP;
END $$;
`, username, username)
`, baseUsername, baseUsername)
_, err = tx.Exec(ctx, defaultPrivilegesSQL)
if err != nil {
@@ -466,7 +548,7 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
// Step 7: Verify user creation before committing
var verifyUsername string
err = tx.QueryRow(ctx, fmt.Sprintf(`SELECT rolname FROM pg_roles WHERE rolname = '%s'`, username)).
err = tx.QueryRow(ctx, fmt.Sprintf(`SELECT rolname FROM pg_roles WHERE rolname = '%s'`, baseUsername)).
Scan(&verifyUsername)
if err != nil {
return "", "", fmt.Errorf("failed to verify user creation: %w", err)
@@ -477,8 +559,15 @@ func (p *PostgresqlDatabase) CreateReadOnlyUser(
}
success = true
logger.Info("Read-only user created successfully", "username", username)
return username, newPassword, nil
// Return connectionUsername (with project ID suffix for Supabase) for the caller to use when connecting
logger.Info(
"Read-only user created successfully",
"username",
baseUsername,
"connectionUsername",
connectionUsername,
)
return connectionUsername, newPassword, nil
}
return "", "", errors.New("failed to generate unique username after 3 attempts")
@@ -521,10 +610,12 @@ func testSingleDatabaseConnection(
}
}()
// Check version after successful connection
if err := verifyDatabaseVersion(ctx, conn, postgresDb.Version); err != nil {
// Detect and set the database version automatically
detectedVersion, err := detectDatabaseVersion(ctx, conn)
if err != nil {
return err
}
postgresDb.Version = detectedVersion
// Test if we can perform basic operations (like pg_dump would need)
if err := testBasicOperations(ctx, conn, *postgresDb.Database); err != nil {
@@ -538,35 +629,31 @@ func testSingleDatabaseConnection(
return nil
}
// verifyDatabaseVersion checks if the actual database version matches the specified version
func verifyDatabaseVersion(
ctx context.Context,
conn *pgx.Conn,
expectedVersion tools.PostgresqlVersion,
) error {
// detectDatabaseVersion queries and returns the PostgreSQL major version
func detectDatabaseVersion(ctx context.Context, conn *pgx.Conn) (tools.PostgresqlVersion, error) {
var versionStr string
err := conn.QueryRow(ctx, "SELECT version()").Scan(&versionStr)
if err != nil {
return fmt.Errorf("failed to query database version: %w", err)
return "", fmt.Errorf("failed to query database version: %w", err)
}
// Parse version from string like "PostgreSQL 14.2 on x86_64-pc-linux-gnu..."
re := regexp.MustCompile(`PostgreSQL (\d+)\.`)
// or "PostgreSQL 16 maintained by Postgre BY..." (some builds omit minor version)
re := regexp.MustCompile(`PostgreSQL (\d+)`)
matches := re.FindStringSubmatch(versionStr)
if len(matches) < 2 {
return fmt.Errorf("could not parse version from: %s", versionStr)
return "", fmt.Errorf("could not parse version from: %s", versionStr)
}
actualVersion := tools.GetPostgresqlVersionEnum(matches[1])
if actualVersion != expectedVersion {
return fmt.Errorf(
"you specified wrong version. Real version is %s, but you specified %s",
actualVersion,
expectedVersion,
)
}
majorVersion := matches[1]
return nil
// Map to known PostgresqlVersion enum values
switch majorVersion {
case "12", "13", "14", "15", "16", "17", "18":
return tools.PostgresqlVersion(majorVersion), nil
default:
return "", fmt.Errorf("unsupported PostgreSQL version: %s", majorVersion)
}
}
// testBasicOperations tests basic operations that backup tools need
@@ -593,7 +680,8 @@ func buildConnectionStringForDB(p *PostgresqlDatabase, dbName string, password s
sslMode = "require"
}
return fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=%s",
return fmt.Sprintf(
"host=%s port=%d user=%s password=%s dbname=%s sslmode=%s default_query_exec_mode=simple_protocol standard_conforming_strings=on client_encoding=UTF8",
p.Host,
p.Port,
p.Username,
@@ -613,3 +701,15 @@ func decryptPasswordIfNeeded(
}
return encryptor.Decrypt(databaseID, password)
}
func isSupabaseConnection(host, username string) bool {
return strings.Contains(strings.ToLower(host), "supabase") ||
strings.Contains(strings.ToLower(username), "supabase")
}
func extractSupabaseProjectID(username string) string {
if idx := strings.Index(username, "."); idx != -1 {
return username[idx+1:]
}
return ""
}

View File

@@ -246,6 +246,188 @@ func Test_ReadOnlyUser_MultipleSchemas_AllAccessible(t *testing.T) {
assert.NoError(t, err)
}
func Test_CreateReadOnlyUser_DatabaseNameWithDash_Success(t *testing.T) {
env := config.GetEnv()
container := connectToPostgresContainer(t, env.TestPostgres16Port)
defer container.DB.Close()
dashDbName := "test-db-with-dash"
_, err := container.DB.Exec(fmt.Sprintf(`DROP DATABASE IF EXISTS "%s"`, dashDbName))
assert.NoError(t, err)
_, err = container.DB.Exec(fmt.Sprintf(`CREATE DATABASE "%s"`, dashDbName))
assert.NoError(t, err)
defer func() {
_, _ = container.DB.Exec(fmt.Sprintf(`DROP DATABASE IF EXISTS "%s"`, dashDbName))
}()
dashDSN := fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=disable",
container.Host, container.Port, container.Username, container.Password, dashDbName)
dashDB, err := sqlx.Connect("postgres", dashDSN)
assert.NoError(t, err)
defer dashDB.Close()
_, err = dashDB.Exec(`
CREATE TABLE dash_test (
id SERIAL PRIMARY KEY,
data TEXT NOT NULL
);
INSERT INTO dash_test (data) VALUES ('test1'), ('test2');
`)
assert.NoError(t, err)
pgModel := &PostgresqlDatabase{
Version: tools.GetPostgresqlVersionEnum("16"),
Host: container.Host,
Port: container.Port,
Username: container.Username,
Password: container.Password,
Database: &dashDbName,
IsHttps: false,
}
logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
ctx := context.Background()
username, password, err := pgModel.CreateReadOnlyUser(ctx, logger, nil, uuid.New())
assert.NoError(t, err)
assert.NotEmpty(t, username)
assert.NotEmpty(t, password)
assert.True(t, strings.HasPrefix(username, "postgresus-"))
readOnlyDSN := fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=disable",
container.Host, container.Port, username, password, dashDbName)
readOnlyConn, err := sqlx.Connect("postgres", readOnlyDSN)
assert.NoError(t, err)
defer readOnlyConn.Close()
var count int
err = readOnlyConn.Get(&count, "SELECT COUNT(*) FROM dash_test")
assert.NoError(t, err)
assert.Equal(t, 2, count)
_, err = readOnlyConn.Exec("INSERT INTO dash_test (data) VALUES ('should-fail')")
assert.Error(t, err)
assert.Contains(t, err.Error(), "permission denied")
_, err = dashDB.Exec(fmt.Sprintf(`DROP OWNED BY "%s" CASCADE`, username))
if err != nil {
t.Logf("Warning: Failed to drop owned objects: %v", err)
}
_, err = dashDB.Exec(fmt.Sprintf(`DROP USER IF EXISTS "%s"`, username))
assert.NoError(t, err)
}
func Test_CreateReadOnlyUser_Supabase_UserCanReadButNotWrite(t *testing.T) {
env := config.GetEnv()
if env.TestSupabaseHost == "" {
t.Skip("Skipping Supabase test: missing environment variables")
}
portInt, err := strconv.Atoi(env.TestSupabasePort)
assert.NoError(t, err)
dsn := fmt.Sprintf(
"host=%s port=%d user=%s password=%s dbname=%s sslmode=require",
env.TestSupabaseHost,
portInt,
env.TestSupabaseUsername,
env.TestSupabasePassword,
env.TestSupabaseDatabase,
)
adminDB, err := sqlx.Connect("postgres", dsn)
assert.NoError(t, err)
defer adminDB.Close()
tableName := fmt.Sprintf(
"readonly_test_%s",
strings.ReplaceAll(uuid.New().String()[:8], "-", ""),
)
_, err = adminDB.Exec(fmt.Sprintf(`
DROP TABLE IF EXISTS public.%s CASCADE;
CREATE TABLE public.%s (
id SERIAL PRIMARY KEY,
data TEXT NOT NULL
);
INSERT INTO public.%s (data) VALUES ('test1'), ('test2');
`, tableName, tableName, tableName))
assert.NoError(t, err)
defer func() {
_, _ = adminDB.Exec(fmt.Sprintf(`DROP TABLE IF EXISTS public.%s CASCADE`, tableName))
}()
pgModel := &PostgresqlDatabase{
Host: env.TestSupabaseHost,
Port: portInt,
Username: env.TestSupabaseUsername,
Password: env.TestSupabasePassword,
Database: &env.TestSupabaseDatabase,
IsHttps: true,
}
logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
ctx := context.Background()
connectionUsername, newPassword, err := pgModel.CreateReadOnlyUser(ctx, logger, nil, uuid.New())
assert.NoError(t, err)
assert.NotEmpty(t, connectionUsername)
assert.NotEmpty(t, newPassword)
assert.True(t, strings.HasPrefix(connectionUsername, "postgresus-"))
baseUsername := connectionUsername
if idx := strings.Index(connectionUsername, "."); idx != -1 {
baseUsername = connectionUsername[:idx]
}
defer func() {
_, _ = adminDB.Exec(fmt.Sprintf(`DROP OWNED BY "%s" CASCADE`, baseUsername))
_, _ = adminDB.Exec(fmt.Sprintf(`DROP USER IF EXISTS "%s"`, baseUsername))
}()
readOnlyDSN := fmt.Sprintf(
"host=%s port=%d user=%s password=%s dbname=%s sslmode=require",
env.TestSupabaseHost,
portInt,
connectionUsername,
newPassword,
env.TestSupabaseDatabase,
)
readOnlyConn, err := sqlx.Connect("postgres", readOnlyDSN)
assert.NoError(t, err)
defer readOnlyConn.Close()
var count int
err = readOnlyConn.Get(&count, fmt.Sprintf("SELECT COUNT(*) FROM public.%s", tableName))
assert.NoError(t, err)
assert.Equal(t, 2, count)
_, err = readOnlyConn.Exec(
fmt.Sprintf("INSERT INTO public.%s (data) VALUES ('should-fail')", tableName),
)
assert.Error(t, err)
assert.Contains(t, err.Error(), "permission denied")
_, err = readOnlyConn.Exec(
fmt.Sprintf("UPDATE public.%s SET data = 'hacked' WHERE id = 1", tableName),
)
assert.Error(t, err)
assert.Contains(t, err.Error(), "permission denied")
_, err = readOnlyConn.Exec(fmt.Sprintf("DELETE FROM public.%s WHERE id = 1", tableName))
assert.Error(t, err)
assert.Contains(t, err.Error(), "permission denied")
_, err = readOnlyConn.Exec("CREATE TABLE public.hack_table (id INT)")
assert.Error(t, err)
assert.Contains(t, err.Error(), "permission denied")
}
type PostgresContainer struct {
Host string
Port int

View File

@@ -75,6 +75,16 @@ func (d *Database) EncryptSensitiveFields(encryptor encryption.FieldEncryptor) e
return nil
}
func (d *Database) PopulateVersionIfEmpty(
logger *slog.Logger,
encryptor encryption.FieldEncryptor,
) error {
if d.Postgresql != nil {
return d.Postgresql.PopulateVersionIfEmpty(logger, encryptor, d.ID)
}
return nil
}
func (d *Database) Update(incoming *Database) {
d.Name = incoming.Name
d.Type = incoming.Type

View File

@@ -68,6 +68,10 @@ func (s *DatabaseService) CreateDatabase(
return nil, err
}
if err := database.PopulateVersionIfEmpty(s.logger, s.fieldEncryptor); err != nil {
return nil, fmt.Errorf("failed to auto-detect database version: %w", err)
}
if err := database.EncryptSensitiveFields(s.fieldEncryptor); err != nil {
return nil, fmt.Errorf("failed to encrypt sensitive fields: %w", err)
}
@@ -125,6 +129,10 @@ func (s *DatabaseService) UpdateDatabase(
return err
}
if err := existingDatabase.PopulateVersionIfEmpty(s.logger, s.fieldEncryptor); err != nil {
return fmt.Errorf("failed to auto-detect database version: %w", err)
}
if err := existingDatabase.EncryptSensitiveFields(s.fieldEncryptor); err != nil {
return fmt.Errorf("failed to encrypt sensitive fields: %w", err)
}

View File

@@ -0,0 +1,28 @@
package email_notifier
import (
"errors"
"net/smtp"
)
type loginAuth struct {
username, password string
}
func (a *loginAuth) Start(server *smtp.ServerInfo) (string, []byte, error) {
return "LOGIN", []byte{}, nil
}
func (a *loginAuth) Next(fromServer []byte, more bool) ([]byte, error) {
if more {
switch string(fromServer) {
case "Username:":
return []byte(a.username), nil
case "Password:":
return []byte(a.password), nil
default:
return nil, errors.New("unknown LOGIN challenge: " + string(fromServer))
}
}
return nil, nil
}

View File

@@ -58,11 +58,10 @@ func (e *EmailNotifier) Validate(encryptor encryption.FieldEncryptor) error {
func (e *EmailNotifier) Send(
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
_ *slog.Logger,
heading string,
message string,
) error {
// Decrypt SMTP password if provided
var smtpPassword string
if e.SMTPPassword != "" {
decrypted, err := encryptor.Decrypt(e.NotifierID, e.SMTPPassword)
@@ -72,7 +71,6 @@ func (e *EmailNotifier) Send(
smtpPassword = decrypted
}
// Compose email
from := e.From
if from == "" {
from = e.SMTPUser
@@ -81,153 +79,13 @@ func (e *EmailNotifier) Send(
}
}
to := []string{e.TargetEmail}
// Format the email content
subject := fmt.Sprintf("Subject: %s\r\n", heading)
mime := fmt.Sprintf(
"MIME-version: 1.0;\nContent-Type: %s; charset=\"%s\";\n\n",
MIMETypeHTML,
MIMECharsetUTF8,
)
body := message
fromHeader := fmt.Sprintf("From: %s\r\n", from)
toHeader := fmt.Sprintf("To: %s\r\n", e.TargetEmail)
// Combine all parts of the email
emailContent := []byte(fromHeader + toHeader + subject + mime + body)
addr := net.JoinHostPort(e.SMTPHost, fmt.Sprintf("%d", e.SMTPPort))
timeout := DefaultTimeout
// Determine if authentication is required
emailContent := e.buildEmailContent(heading, message, from)
isAuthRequired := e.SMTPUser != "" && smtpPassword != ""
// Handle different port scenarios
if e.SMTPPort == ImplicitTLSPort {
// Implicit TLS (port 465)
// Set up TLS config
tlsConfig := &tls.Config{
ServerName: e.SMTPHost,
}
// Dial with timeout
dialer := &net.Dialer{Timeout: timeout}
conn, err := tls.DialWithDialer(dialer, "tcp", addr, tlsConfig)
if err != nil {
return fmt.Errorf("failed to connect to SMTP server: %w", err)
}
defer func() {
_ = conn.Close()
}()
// Create SMTP client
client, err := smtp.NewClient(conn, e.SMTPHost)
if err != nil {
return fmt.Errorf("failed to create SMTP client: %w", err)
}
defer func() {
_ = client.Quit()
}()
// Set up authentication only if credentials are provided
if isAuthRequired {
auth := smtp.PlainAuth("", e.SMTPUser, smtpPassword, e.SMTPHost)
if err := client.Auth(auth); err != nil {
return fmt.Errorf("SMTP authentication failed: %w", err)
}
}
// Set sender and recipients
if err := client.Mail(from); err != nil {
return fmt.Errorf("failed to set sender: %w", err)
}
for _, recipient := range to {
if err := client.Rcpt(recipient); err != nil {
return fmt.Errorf("failed to set recipient: %w", err)
}
}
// Send the email body
writer, err := client.Data()
if err != nil {
return fmt.Errorf("failed to get data writer: %w", err)
}
_, err = writer.Write(emailContent)
if err != nil {
return fmt.Errorf("failed to write email content: %w", err)
}
err = writer.Close()
if err != nil {
return fmt.Errorf("failed to close data writer: %w", err)
}
return nil
} else {
// STARTTLS (port 587) or other ports
// Create a custom dialer with timeout
dialer := &net.Dialer{Timeout: timeout}
conn, err := dialer.Dial("tcp", addr)
if err != nil {
return fmt.Errorf("failed to connect to SMTP server: %w", err)
}
// Create client from connection
client, err := smtp.NewClient(conn, e.SMTPHost)
if err != nil {
return fmt.Errorf("failed to create SMTP client: %w", err)
}
defer func() {
_ = client.Quit()
}()
// Send email using the client
if err := client.Hello(DefaultHelloName); err != nil {
return fmt.Errorf("SMTP hello failed: %w", err)
}
// Start TLS if available
if ok, _ := client.Extension("STARTTLS"); ok {
if err := client.StartTLS(&tls.Config{ServerName: e.SMTPHost}); err != nil {
return fmt.Errorf("STARTTLS failed: %w", err)
}
}
// Authenticate only if credentials are provided
if isAuthRequired {
auth := smtp.PlainAuth("", e.SMTPUser, smtpPassword, e.SMTPHost)
if err := client.Auth(auth); err != nil {
return fmt.Errorf("SMTP authentication failed: %w", err)
}
}
if err := client.Mail(from); err != nil {
return fmt.Errorf("failed to set sender: %w", err)
}
for _, recipient := range to {
if err := client.Rcpt(recipient); err != nil {
return fmt.Errorf("failed to set recipient: %w", err)
}
}
writer, err := client.Data()
if err != nil {
return fmt.Errorf("failed to get data writer: %w", err)
}
_, err = writer.Write(emailContent)
if err != nil {
return fmt.Errorf("failed to write email content: %w", err)
}
err = writer.Close()
if err != nil {
return fmt.Errorf("failed to close data writer: %w", err)
}
return client.Quit()
return e.sendImplicitTLS(emailContent, from, smtpPassword, isAuthRequired)
}
return e.sendStartTLS(emailContent, from, smtpPassword, isAuthRequired)
}
func (e *EmailNotifier) HideSensitiveData() {
@@ -256,3 +114,166 @@ func (e *EmailNotifier) EncryptSensitiveData(encryptor encryption.FieldEncryptor
}
return nil
}
func (e *EmailNotifier) buildEmailContent(heading, message, from string) []byte {
subject := fmt.Sprintf("Subject: %s\r\n", heading)
mime := fmt.Sprintf(
"MIME-version: 1.0;\nContent-Type: %s; charset=\"%s\";\n\n",
MIMETypeHTML,
MIMECharsetUTF8,
)
fromHeader := fmt.Sprintf("From: %s\r\n", from)
toHeader := fmt.Sprintf("To: %s\r\n", e.TargetEmail)
return []byte(fromHeader + toHeader + subject + mime + message)
}
func (e *EmailNotifier) sendImplicitTLS(
emailContent []byte,
from string,
password string,
isAuthRequired bool,
) error {
createClient := func() (*smtp.Client, func(), error) {
return e.createImplicitTLSClient()
}
client, cleanup, err := e.authenticateWithRetry(createClient, password, isAuthRequired)
if err != nil {
return err
}
defer cleanup()
return e.sendEmail(client, from, emailContent)
}
func (e *EmailNotifier) sendStartTLS(
emailContent []byte,
from string,
password string,
isAuthRequired bool,
) error {
createClient := func() (*smtp.Client, func(), error) {
return e.createStartTLSClient()
}
client, cleanup, err := e.authenticateWithRetry(createClient, password, isAuthRequired)
if err != nil {
return err
}
defer cleanup()
return e.sendEmail(client, from, emailContent)
}
func (e *EmailNotifier) createImplicitTLSClient() (*smtp.Client, func(), error) {
addr := net.JoinHostPort(e.SMTPHost, fmt.Sprintf("%d", e.SMTPPort))
tlsConfig := &tls.Config{ServerName: e.SMTPHost}
dialer := &net.Dialer{Timeout: DefaultTimeout}
conn, err := tls.DialWithDialer(dialer, "tcp", addr, tlsConfig)
if err != nil {
return nil, nil, fmt.Errorf("failed to connect to SMTP server: %w", err)
}
client, err := smtp.NewClient(conn, e.SMTPHost)
if err != nil {
_ = conn.Close()
return nil, nil, fmt.Errorf("failed to create SMTP client: %w", err)
}
return client, func() { _ = client.Quit() }, nil
}
func (e *EmailNotifier) createStartTLSClient() (*smtp.Client, func(), error) {
addr := net.JoinHostPort(e.SMTPHost, fmt.Sprintf("%d", e.SMTPPort))
dialer := &net.Dialer{Timeout: DefaultTimeout}
conn, err := dialer.Dial("tcp", addr)
if err != nil {
return nil, nil, fmt.Errorf("failed to connect to SMTP server: %w", err)
}
client, err := smtp.NewClient(conn, e.SMTPHost)
if err != nil {
_ = conn.Close()
return nil, nil, fmt.Errorf("failed to create SMTP client: %w", err)
}
if err := client.Hello(DefaultHelloName); err != nil {
_ = client.Quit()
_ = conn.Close()
return nil, nil, fmt.Errorf("SMTP hello failed: %w", err)
}
if ok, _ := client.Extension("STARTTLS"); ok {
if err := client.StartTLS(&tls.Config{ServerName: e.SMTPHost}); err != nil {
_ = client.Quit()
_ = conn.Close()
return nil, nil, fmt.Errorf("STARTTLS failed: %w", err)
}
}
return client, func() { _ = client.Quit() }, nil
}
func (e *EmailNotifier) authenticateWithRetry(
createClient func() (*smtp.Client, func(), error),
password string,
isAuthRequired bool,
) (*smtp.Client, func(), error) {
client, cleanup, err := createClient()
if err != nil {
return nil, nil, err
}
if !isAuthRequired {
return client, cleanup, nil
}
// Try PLAIN auth first
plainAuth := smtp.PlainAuth("", e.SMTPUser, password, e.SMTPHost)
if err := client.Auth(plainAuth); err == nil {
return client, cleanup, nil
}
// PLAIN auth failed, connection may be closed - recreate and try LOGIN auth
cleanup()
client, cleanup, err = createClient()
if err != nil {
return nil, nil, err
}
loginAuth := &loginAuth{username: e.SMTPUser, password: password}
if err := client.Auth(loginAuth); err != nil {
cleanup()
return nil, nil, fmt.Errorf("SMTP authentication failed: %w", err)
}
return client, cleanup, nil
}
func (e *EmailNotifier) sendEmail(client *smtp.Client, from string, content []byte) error {
if err := client.Mail(from); err != nil {
return fmt.Errorf("failed to set sender: %w", err)
}
if err := client.Rcpt(e.TargetEmail); err != nil {
return fmt.Errorf("failed to set recipient: %w", err)
}
writer, err := client.Data()
if err != nil {
return fmt.Errorf("failed to get data writer: %w", err)
}
if _, err = writer.Write(content); err != nil {
return fmt.Errorf("failed to write email content: %w", err)
}
if err = writer.Close(); err != nil {
return fmt.Errorf("failed to close data writer: %w", err)
}
return nil
}

View File

@@ -10,20 +10,57 @@ import (
"net/http"
"net/url"
"postgresus-backend/internal/util/encryption"
"strings"
"github.com/google/uuid"
"gorm.io/gorm"
)
type WebhookHeader struct {
Key string `json:"key"`
Value string `json:"value"`
}
type WebhookNotifier struct {
NotifierID uuid.UUID `json:"notifierId" gorm:"primaryKey;column:notifier_id"`
WebhookURL string `json:"webhookUrl" gorm:"not null;column:webhook_url"`
WebhookMethod WebhookMethod `json:"webhookMethod" gorm:"not null;column:webhook_method"`
BodyTemplate *string `json:"bodyTemplate" gorm:"column:body_template;type:text"`
HeadersJSON string `json:"-" gorm:"column:headers;type:text"`
Headers []WebhookHeader `json:"headers" gorm:"-"`
}
func (t *WebhookNotifier) TableName() string {
return "webhook_notifiers"
}
func (t *WebhookNotifier) BeforeSave(_ *gorm.DB) error {
if len(t.Headers) > 0 {
data, err := json.Marshal(t.Headers)
if err != nil {
return err
}
t.HeadersJSON = string(data)
} else {
t.HeadersJSON = "[]"
}
return nil
}
func (t *WebhookNotifier) AfterFind(_ *gorm.DB) error {
if t.HeadersJSON != "" {
if err := json.Unmarshal([]byte(t.HeadersJSON), &t.Headers); err != nil {
return err
}
}
return nil
}
func (t *WebhookNotifier) Validate(encryptor encryption.FieldEncryptor) error {
if t.WebhookURL == "" {
return errors.New("webhook URL is required")
@@ -49,66 +86,9 @@ func (t *WebhookNotifier) Send(
switch t.WebhookMethod {
case WebhookMethodGET:
reqURL := fmt.Sprintf("%s?heading=%s&message=%s",
webhookURL,
url.QueryEscape(heading),
url.QueryEscape(message),
)
resp, err := http.Get(reqURL)
if err != nil {
return fmt.Errorf("failed to send GET webhook: %w", err)
}
defer func() {
if cerr := resp.Body.Close(); cerr != nil {
logger.Error("failed to close response body", "error", cerr)
}
}()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
body, _ := io.ReadAll(resp.Body)
return fmt.Errorf(
"webhook GET returned status: %s, body: %s",
resp.Status,
string(body),
)
}
return nil
return t.sendGET(webhookURL, heading, message, logger)
case WebhookMethodPOST:
payload := map[string]string{
"heading": heading,
"message": message,
}
body, err := json.Marshal(payload)
if err != nil {
return fmt.Errorf("failed to marshal webhook payload: %w", err)
}
resp, err := http.Post(webhookURL, "application/json", bytes.NewReader(body))
if err != nil {
return fmt.Errorf("failed to send POST webhook: %w", err)
}
defer func() {
if cerr := resp.Body.Close(); cerr != nil {
logger.Error("failed to close response body", "error", cerr)
}
}()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
body, _ := io.ReadAll(resp.Body)
return fmt.Errorf(
"webhook POST returned status: %s, body: %s",
resp.Status,
string(body),
)
}
return nil
return t.sendPOST(webhookURL, heading, message, logger)
default:
return fmt.Errorf("unsupported webhook method: %s", t.WebhookMethod)
}
@@ -120,15 +100,144 @@ func (t *WebhookNotifier) HideSensitiveData() {
func (t *WebhookNotifier) Update(incoming *WebhookNotifier) {
t.WebhookURL = incoming.WebhookURL
t.WebhookMethod = incoming.WebhookMethod
t.BodyTemplate = incoming.BodyTemplate
t.Headers = incoming.Headers
}
func (t *WebhookNotifier) EncryptSensitiveData(encryptor encryption.FieldEncryptor) error {
if t.WebhookURL != "" {
encrypted, err := encryptor.Encrypt(t.NotifierID, t.WebhookURL)
if err != nil {
return fmt.Errorf("failed to encrypt webhook URL: %w", err)
}
t.WebhookURL = encrypted
}
return nil
}
func (t *WebhookNotifier) sendGET(webhookURL, heading, message string, logger *slog.Logger) error {
reqURL := fmt.Sprintf("%s?heading=%s&message=%s",
webhookURL,
url.QueryEscape(heading),
url.QueryEscape(message),
)
req, err := http.NewRequest(http.MethodGet, reqURL, nil)
if err != nil {
return fmt.Errorf("failed to create GET request: %w", err)
}
t.applyHeaders(req)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("failed to send GET webhook: %w", err)
}
defer func() {
if cerr := resp.Body.Close(); cerr != nil {
logger.Error("failed to close response body", "error", cerr)
}
}()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
body, _ := io.ReadAll(resp.Body)
return fmt.Errorf(
"webhook GET returned status: %s, body: %s",
resp.Status,
string(body),
)
}
return nil
}
func (t *WebhookNotifier) sendPOST(webhookURL, heading, message string, logger *slog.Logger) error {
body := t.buildRequestBody(heading, message)
req, err := http.NewRequest(http.MethodPost, webhookURL, bytes.NewReader(body))
if err != nil {
return fmt.Errorf("failed to create POST request: %w", err)
}
hasContentType := false
for _, h := range t.Headers {
if strings.EqualFold(h.Key, "Content-Type") {
hasContentType = true
break
}
}
if !hasContentType {
req.Header.Set("Content-Type", "application/json")
}
t.applyHeaders(req)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("failed to send POST webhook: %w", err)
}
defer func() {
if cerr := resp.Body.Close(); cerr != nil {
logger.Error("failed to close response body", "error", cerr)
}
}()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
respBody, _ := io.ReadAll(resp.Body)
return fmt.Errorf(
"webhook POST returned status: %s, body: %s",
resp.Status,
string(respBody),
)
}
return nil
}
func (t *WebhookNotifier) buildRequestBody(heading, message string) []byte {
if t.BodyTemplate != nil && *t.BodyTemplate != "" {
result := *t.BodyTemplate
result = strings.ReplaceAll(result, "{{heading}}", escapeJSONString(heading))
result = strings.ReplaceAll(result, "{{message}}", escapeJSONString(message))
return []byte(result)
}
payload := map[string]string{
"heading": heading,
"message": message,
}
body, _ := json.Marshal(payload)
return body
}
func (t *WebhookNotifier) applyHeaders(req *http.Request) {
for _, h := range t.Headers {
if h.Key != "" {
req.Header.Set(h.Key, h.Value)
}
}
}
func escapeJSONString(s string) string {
b, err := json.Marshal(s)
if err != nil || len(b) < 2 {
escaped := strings.ReplaceAll(s, `\`, `\\`)
escaped = strings.ReplaceAll(escaped, `"`, `\"`)
escaped = strings.ReplaceAll(escaped, "\n", `\n`)
escaped = strings.ReplaceAll(escaped, "\r", `\r`)
escaped = strings.ReplaceAll(escaped, "\t", `\t`)
return escaped
}
return string(b[1 : len(b)-1])
}

View File

@@ -1,6 +1,7 @@
package restores
import (
"context"
"encoding/json"
"fmt"
"io"
@@ -170,6 +171,36 @@ func Test_RestoreBackup_WhenUserIsNotWorkspaceMember_ReturnsForbidden(t *testing
assert.Contains(t, string(testResp.Body), "insufficient permissions")
}
func Test_RestoreBackup_WithIsExcludeExtensions_FlagPassedCorrectly(t *testing.T) {
router := createTestRouter()
owner := users_testing.CreateTestUser(users_enums.UserRoleMember)
workspace := workspaces_testing.CreateTestWorkspace("Test Workspace", owner, router)
_, backup := createTestDatabaseWithBackupForRestore(workspace, owner, router)
request := RestoreBackupRequest{
PostgresqlDatabase: &postgresql.PostgresqlDatabase{
Version: tools.PostgresqlVersion16,
Host: "localhost",
Port: 5432,
Username: "postgres",
Password: "postgres",
IsExcludeExtensions: true,
},
}
testResp := test_utils.MakePostRequest(
t,
router,
fmt.Sprintf("/api/v1/restores/%s/restore", backup.ID.String()),
"Bearer "+owner.Token,
request,
http.StatusOK,
)
assert.Contains(t, string(testResp.Body), "restore started successfully")
}
func Test_RestoreBackup_AuditLogWritten(t *testing.T) {
router := createTestRouter()
owner := users_testing.CreateTestUser(users_enums.UserRoleMember)
@@ -340,7 +371,7 @@ func createTestBackup(
dummyContent := []byte("dummy backup content for testing")
reader := strings.NewReader(string(dummyContent))
logger := slog.New(slog.NewTextHandler(io.Discard, nil))
if err := storages[0].SaveFile(fieldEncryptor, logger, backup.ID, reader); err != nil {
if err := storages[0].SaveFile(context.Background(), fieldEncryptor, logger, backup.ID, reader); err != nil {
panic(fmt.Sprintf("Failed to create test backup file: %v", err))
}

View File

@@ -8,6 +8,7 @@ import (
"postgresus-backend/internal/features/restores/usecases"
"postgresus-backend/internal/features/storages"
workspaces_services "postgresus-backend/internal/features/workspaces/services"
"postgresus-backend/internal/util/encryption"
"postgresus-backend/internal/util/logger"
)
@@ -22,6 +23,7 @@ var restoreService = &RestoreService{
logger.GetLogger(),
workspaces_services.GetWorkspaceService(),
audit_logs.GetAuditLogService(),
encryption.GetFieldEncryptor(),
}
var restoreController = &RestoreController{
restoreService,

View File

@@ -14,6 +14,7 @@ import (
"postgresus-backend/internal/features/storages"
users_models "postgresus-backend/internal/features/users/models"
workspaces_services "postgresus-backend/internal/features/workspaces/services"
"postgresus-backend/internal/util/encryption"
"postgresus-backend/internal/util/tools"
"time"
@@ -30,6 +31,7 @@ type RestoreService struct {
logger *slog.Logger
workspaceService *workspaces_services.WorkspaceService
auditLogService *audit_logs.AuditLogService
fieldEncryptor encryption.FieldEncryptor
}
func (s *RestoreService) OnBeforeBackupRemove(backup *backups.Backup) error {
@@ -120,12 +122,6 @@ func (s *RestoreService) RestoreBackupWithAuth(
return err
}
fmt.Printf(
"restore from %s to %s\n",
backupDatabase.Postgresql.Version,
requestDTO.PostgresqlDatabase.Version,
)
if tools.IsBackupDbVersionHigherThanRestoreDbVersion(
backupDatabase.Postgresql.Version,
requestDTO.PostgresqlDatabase.Version,
@@ -214,6 +210,15 @@ func (s *RestoreService) RestoreBackup(
Postgresql: requestDTO.PostgresqlDatabase,
}
if err := restoringToDB.PopulateVersionIfEmpty(s.logger, s.fieldEncryptor); err != nil {
return fmt.Errorf("failed to auto-detect database version: %w", err)
}
isExcludeExtensions := false
if requestDTO.PostgresqlDatabase != nil {
isExcludeExtensions = requestDTO.PostgresqlDatabase.IsExcludeExtensions
}
err = s.restoreBackupUsecase.Execute(
backupConfig,
restore,
@@ -221,6 +226,7 @@ func (s *RestoreService) RestoreBackup(
restoringToDB,
backup,
storage,
isExcludeExtensions,
)
if err != nil {
errMsg := err.Error()

View File

@@ -42,6 +42,7 @@ func (uc *RestorePostgresqlBackupUsecase) Execute(
restore models.Restore,
backup *backups.Backup,
storage *storages.Storage,
isExcludeExtensions bool,
) error {
if originalDB.Type != databases.DatabaseTypePostgres {
return errors.New("database type not supported")
@@ -96,6 +97,7 @@ func (uc *RestorePostgresqlBackupUsecase) Execute(
backup,
storage,
pg,
isExcludeExtensions,
)
}
@@ -108,6 +110,7 @@ func (uc *RestorePostgresqlBackupUsecase) restoreFromStorage(
backup *backups.Backup,
storage *storages.Storage,
pgConfig *pgtypes.PostgresqlDatabase,
isExcludeExtensions bool,
) error {
uc.logger.Info(
"Restoring PostgreSQL backup from storage via temporary file",
@@ -115,6 +118,8 @@ func (uc *RestorePostgresqlBackupUsecase) restoreFromStorage(
pgBin,
"args",
args,
"isExcludeExtensions",
isExcludeExtensions,
)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Minute)
@@ -171,6 +176,26 @@ func (uc *RestorePostgresqlBackupUsecase) restoreFromStorage(
}
defer cleanupFunc()
// If excluding extensions, generate filtered TOC list and use it
if isExcludeExtensions {
tocListFile, err := uc.generateFilteredTocList(
ctx,
pgBin,
tempBackupFile,
pgpassFile,
pgConfig,
)
if err != nil {
return fmt.Errorf("failed to generate filtered TOC list: %w", err)
}
defer func() {
_ = os.Remove(tocListFile)
}()
// Add -L flag to use the filtered list
args = append(args, "-L", tocListFile)
}
// Add the temporary backup file as the last argument to pg_restore
args = append(args, tempBackupFile)
@@ -378,7 +403,6 @@ func (uc *RestorePostgresqlBackupUsecase) setupPgRestoreEnvironment(
// Add encoding-related environment variables
cmd.Env = append(cmd.Env, "LC_ALL=C.UTF-8")
cmd.Env = append(cmd.Env, "LANG=C.UTF-8")
cmd.Env = append(cmd.Env, "PGOPTIONS=--client-encoding=UTF8")
shouldRequireSSL := pgConfig.IsHttps
@@ -503,7 +527,7 @@ func (uc *RestorePostgresqlBackupUsecase) copyWithShutdownCheck(
dst io.Writer,
src io.Reader,
) (int64, error) {
buf := make([]byte, 32*1024) // 32KB buffer
buf := make([]byte, 16*1024*1024) // 16MB buffer
var totalBytesWritten int64
for {
@@ -555,6 +579,75 @@ func containsIgnoreCase(str, substr string) bool {
return strings.Contains(strings.ToLower(str), strings.ToLower(substr))
}
// generateFilteredTocList generates a pg_restore TOC list file with extensions filtered out.
// This is used when isExcludeExtensions is true to skip CREATE EXTENSION statements.
func (uc *RestorePostgresqlBackupUsecase) generateFilteredTocList(
ctx context.Context,
pgBin string,
backupFile string,
pgpassFile string,
pgConfig *pgtypes.PostgresqlDatabase,
) (string, error) {
uc.logger.Info("Generating filtered TOC list to exclude extensions", "backupFile", backupFile)
// Run pg_restore -l to get the TOC list
listCmd := exec.CommandContext(ctx, pgBin, "-l", backupFile)
uc.setupPgRestoreEnvironment(listCmd, pgpassFile, pgConfig)
tocOutput, err := listCmd.Output()
if err != nil {
return "", fmt.Errorf("failed to generate TOC list: %w", err)
}
// Filter out EXTENSION-related lines (both CREATE EXTENSION and COMMENT ON EXTENSION)
var filteredLines []string
for line := range strings.SplitSeq(string(tocOutput), "\n") {
trimmedLine := strings.TrimSpace(line)
if trimmedLine == "" {
continue
}
upperLine := strings.ToUpper(trimmedLine)
// Skip lines that contain " EXTENSION " - this catches both:
// - CREATE EXTENSION entries: "3420; 0 0 EXTENSION - uuid-ossp"
// - COMMENT ON EXTENSION entries: "3462; 0 0 COMMENT - EXTENSION "uuid-ossp""
if strings.Contains(upperLine, " EXTENSION ") {
uc.logger.Info("Excluding extension-related entry from restore", "tocLine", trimmedLine)
continue
}
filteredLines = append(filteredLines, line)
}
// Write filtered TOC to temporary file
tocFile, err := os.CreateTemp("", "pg_restore_toc_*.list")
if err != nil {
return "", fmt.Errorf("failed to create TOC list file: %w", err)
}
tocFilePath := tocFile.Name()
filteredContent := strings.Join(filteredLines, "\n")
if _, err := tocFile.WriteString(filteredContent); err != nil {
_ = tocFile.Close()
_ = os.Remove(tocFilePath)
return "", fmt.Errorf("failed to write TOC list file: %w", err)
}
if err := tocFile.Close(); err != nil {
_ = os.Remove(tocFilePath)
return "", fmt.Errorf("failed to close TOC list file: %w", err)
}
uc.logger.Info("Generated filtered TOC list file",
"tocFile", tocFilePath,
"originalLines", len(strings.Split(string(tocOutput), "\n")),
"filteredLines", len(filteredLines),
)
return tocFilePath, nil
}
// createTempPgpassFile creates a temporary .pgpass file with the given password
func (uc *RestorePostgresqlBackupUsecase) createTempPgpassFile(
pgConfig *pgtypes.PostgresqlDatabase,
@@ -564,11 +657,15 @@ func (uc *RestorePostgresqlBackupUsecase) createTempPgpassFile(
return "", nil
}
escapedHost := tools.EscapePgpassField(pgConfig.Host)
escapedUsername := tools.EscapePgpassField(pgConfig.Username)
escapedPassword := tools.EscapePgpassField(password)
pgpassContent := fmt.Sprintf("%s:%d:*:%s:%s",
pgConfig.Host,
escapedHost,
pgConfig.Port,
pgConfig.Username,
password,
escapedUsername,
escapedPassword,
)
tempDir, err := os.MkdirTemp("", "pgpass")

View File

@@ -21,6 +21,7 @@ func (uc *RestoreBackupUsecase) Execute(
restoringToDB *databases.Database,
backup *backups.Backup,
storage *storages.Storage,
isExcludeExtensions bool,
) error {
if originalDB.Type == databases.DatabaseTypePostgres {
return uc.restorePostgresqlBackupUsecase.Execute(
@@ -30,6 +31,7 @@ func (uc *RestoreBackupUsecase) Execute(
restore,
backup,
storage,
isExcludeExtensions,
)
}

View File

@@ -8,10 +8,13 @@ import (
audit_logs "postgresus-backend/internal/features/audit_logs"
azure_blob_storage "postgresus-backend/internal/features/storages/models/azure_blob"
ftp_storage "postgresus-backend/internal/features/storages/models/ftp"
google_drive_storage "postgresus-backend/internal/features/storages/models/google_drive"
local_storage "postgresus-backend/internal/features/storages/models/local"
nas_storage "postgresus-backend/internal/features/storages/models/nas"
rclone_storage "postgresus-backend/internal/features/storages/models/rclone"
s3_storage "postgresus-backend/internal/features/storages/models/s3"
sftp_storage "postgresus-backend/internal/features/storages/models/sftp"
users_enums "postgresus-backend/internal/features/users/enums"
users_middleware "postgresus-backend/internal/features/users/middleware"
users_services "postgresus-backend/internal/features/users/services"
@@ -738,6 +741,155 @@ func Test_StorageSensitiveDataLifecycle_AllTypes(t *testing.T) {
assert.Equal(t, "", storage.GoogleDriveStorage.TokenJSON)
},
},
{
name: "FTP Storage",
storageType: StorageTypeFTP,
createStorage: func(workspaceID uuid.UUID) *Storage {
return &Storage{
WorkspaceID: workspaceID,
Type: StorageTypeFTP,
Name: "Test FTP Storage",
FTPStorage: &ftp_storage.FTPStorage{
Host: "ftp.example.com",
Port: 21,
Username: "testuser",
Password: "original-password",
UseSSL: false,
Path: "/backups",
},
}
},
updateStorage: func(workspaceID uuid.UUID, storageID uuid.UUID) *Storage {
return &Storage{
ID: storageID,
WorkspaceID: workspaceID,
Type: StorageTypeFTP,
Name: "Updated FTP Storage",
FTPStorage: &ftp_storage.FTPStorage{
Host: "ftp2.example.com",
Port: 2121,
Username: "testuser2",
Password: "",
UseSSL: true,
Path: "/backups2",
},
}
},
verifySensitiveData: func(t *testing.T, storage *Storage) {
assert.True(t, strings.HasPrefix(storage.FTPStorage.Password, "enc:"),
"Password should be encrypted with 'enc:' prefix")
encryptor := encryption.GetFieldEncryptor()
password, err := encryptor.Decrypt(storage.ID, storage.FTPStorage.Password)
assert.NoError(t, err)
assert.Equal(t, "original-password", password)
},
verifyHiddenData: func(t *testing.T, storage *Storage) {
assert.Equal(t, "", storage.FTPStorage.Password)
},
},
{
name: "SFTP Storage",
storageType: StorageTypeSFTP,
createStorage: func(workspaceID uuid.UUID) *Storage {
return &Storage{
WorkspaceID: workspaceID,
Type: StorageTypeSFTP,
Name: "Test SFTP Storage",
SFTPStorage: &sftp_storage.SFTPStorage{
Host: "sftp.example.com",
Port: 22,
Username: "testuser",
Password: "original-password",
PrivateKey: "original-private-key",
SkipHostKeyVerify: false,
Path: "/backups",
},
}
},
updateStorage: func(workspaceID uuid.UUID, storageID uuid.UUID) *Storage {
return &Storage{
ID: storageID,
WorkspaceID: workspaceID,
Type: StorageTypeSFTP,
Name: "Updated SFTP Storage",
SFTPStorage: &sftp_storage.SFTPStorage{
Host: "sftp2.example.com",
Port: 2222,
Username: "testuser2",
Password: "",
PrivateKey: "",
SkipHostKeyVerify: true,
Path: "/backups2",
},
}
},
verifySensitiveData: func(t *testing.T, storage *Storage) {
assert.True(t, strings.HasPrefix(storage.SFTPStorage.Password, "enc:"),
"Password should be encrypted with 'enc:' prefix")
assert.True(t, strings.HasPrefix(storage.SFTPStorage.PrivateKey, "enc:"),
"PrivateKey should be encrypted with 'enc:' prefix")
encryptor := encryption.GetFieldEncryptor()
password, err := encryptor.Decrypt(storage.ID, storage.SFTPStorage.Password)
assert.NoError(t, err)
assert.Equal(t, "original-password", password)
privateKey, err := encryptor.Decrypt(storage.ID, storage.SFTPStorage.PrivateKey)
assert.NoError(t, err)
assert.Equal(t, "original-private-key", privateKey)
},
verifyHiddenData: func(t *testing.T, storage *Storage) {
assert.Equal(t, "", storage.SFTPStorage.Password)
assert.Equal(t, "", storage.SFTPStorage.PrivateKey)
},
},
{
name: "Rclone Storage",
storageType: StorageTypeRclone,
createStorage: func(workspaceID uuid.UUID) *Storage {
return &Storage{
WorkspaceID: workspaceID,
Type: StorageTypeRclone,
Name: "Test Rclone Storage",
RcloneStorage: &rclone_storage.RcloneStorage{
ConfigContent: "[myremote]\ntype = s3\nprovider = AWS\naccess_key_id = test\nsecret_access_key = secret\n",
RemotePath: "/backups",
},
}
},
updateStorage: func(workspaceID uuid.UUID, storageID uuid.UUID) *Storage {
return &Storage{
ID: storageID,
WorkspaceID: workspaceID,
Type: StorageTypeRclone,
Name: "Updated Rclone Storage",
RcloneStorage: &rclone_storage.RcloneStorage{
ConfigContent: "",
RemotePath: "/backups2",
},
}
},
verifySensitiveData: func(t *testing.T, storage *Storage) {
assert.True(t, strings.HasPrefix(storage.RcloneStorage.ConfigContent, "enc:"),
"ConfigContent should be encrypted with 'enc:' prefix")
encryptor := encryption.GetFieldEncryptor()
configContent, err := encryptor.Decrypt(
storage.ID,
storage.RcloneStorage.ConfigContent,
)
assert.NoError(t, err)
assert.Equal(
t,
"[myremote]\ntype = s3\nprovider = AWS\naccess_key_id = test\nsecret_access_key = secret\n",
configContent,
)
},
verifyHiddenData: func(t *testing.T, storage *Storage) {
assert.Equal(t, "", storage.RcloneStorage.ConfigContent)
},
},
}
for _, tc := range testCases {

View File

@@ -8,4 +8,7 @@ const (
StorageTypeGoogleDrive StorageType = "GOOGLE_DRIVE"
StorageTypeNAS StorageType = "NAS"
StorageTypeAzureBlob StorageType = "AZURE_BLOB"
StorageTypeFTP StorageType = "FTP"
StorageTypeSFTP StorageType = "SFTP"
StorageTypeRclone StorageType = "RCLONE"
)

View File

@@ -1,6 +1,7 @@
package storages
import (
"context"
"io"
"log/slog"
"postgresus-backend/internal/util/encryption"
@@ -10,6 +11,7 @@ import (
type StorageFileSaver interface {
SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,

View File

@@ -1,14 +1,18 @@
package storages
import (
"context"
"errors"
"io"
"log/slog"
azure_blob_storage "postgresus-backend/internal/features/storages/models/azure_blob"
ftp_storage "postgresus-backend/internal/features/storages/models/ftp"
google_drive_storage "postgresus-backend/internal/features/storages/models/google_drive"
local_storage "postgresus-backend/internal/features/storages/models/local"
nas_storage "postgresus-backend/internal/features/storages/models/nas"
rclone_storage "postgresus-backend/internal/features/storages/models/rclone"
s3_storage "postgresus-backend/internal/features/storages/models/s3"
sftp_storage "postgresus-backend/internal/features/storages/models/sftp"
"postgresus-backend/internal/util/encryption"
"github.com/google/uuid"
@@ -27,15 +31,19 @@ type Storage struct {
GoogleDriveStorage *google_drive_storage.GoogleDriveStorage `json:"googleDriveStorage" gorm:"foreignKey:StorageID"`
NASStorage *nas_storage.NASStorage `json:"nasStorage" gorm:"foreignKey:StorageID"`
AzureBlobStorage *azure_blob_storage.AzureBlobStorage `json:"azureBlobStorage" gorm:"foreignKey:StorageID"`
FTPStorage *ftp_storage.FTPStorage `json:"ftpStorage" gorm:"foreignKey:StorageID"`
SFTPStorage *sftp_storage.SFTPStorage `json:"sftpStorage" gorm:"foreignKey:StorageID"`
RcloneStorage *rclone_storage.RcloneStorage `json:"rcloneStorage" gorm:"foreignKey:StorageID"`
}
func (s *Storage) SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,
file io.Reader,
) error {
err := s.getSpecificStorage().SaveFile(encryptor, logger, fileID, file)
err := s.getSpecificStorage().SaveFile(ctx, encryptor, logger, fileID, file)
if err != nil {
lastSaveError := err.Error()
s.LastSaveError = &lastSaveError
@@ -107,6 +115,18 @@ func (s *Storage) Update(incoming *Storage) {
if s.AzureBlobStorage != nil && incoming.AzureBlobStorage != nil {
s.AzureBlobStorage.Update(incoming.AzureBlobStorage)
}
case StorageTypeFTP:
if s.FTPStorage != nil && incoming.FTPStorage != nil {
s.FTPStorage.Update(incoming.FTPStorage)
}
case StorageTypeSFTP:
if s.SFTPStorage != nil && incoming.SFTPStorage != nil {
s.SFTPStorage.Update(incoming.SFTPStorage)
}
case StorageTypeRclone:
if s.RcloneStorage != nil && incoming.RcloneStorage != nil {
s.RcloneStorage.Update(incoming.RcloneStorage)
}
}
}
@@ -122,6 +142,12 @@ func (s *Storage) getSpecificStorage() StorageFileSaver {
return s.NASStorage
case StorageTypeAzureBlob:
return s.AzureBlobStorage
case StorageTypeFTP:
return s.FTPStorage
case StorageTypeSFTP:
return s.SFTPStorage
case StorageTypeRclone:
return s.RcloneStorage
default:
panic("invalid storage type: " + string(s.Type))
}

View File

@@ -9,10 +9,13 @@ import (
"path/filepath"
"postgresus-backend/internal/config"
azure_blob_storage "postgresus-backend/internal/features/storages/models/azure_blob"
ftp_storage "postgresus-backend/internal/features/storages/models/ftp"
google_drive_storage "postgresus-backend/internal/features/storages/models/google_drive"
local_storage "postgresus-backend/internal/features/storages/models/local"
nas_storage "postgresus-backend/internal/features/storages/models/nas"
rclone_storage "postgresus-backend/internal/features/storages/models/rclone"
s3_storage "postgresus-backend/internal/features/storages/models/s3"
sftp_storage "postgresus-backend/internal/features/storages/models/sftp"
"postgresus-backend/internal/util/encryption"
"postgresus-backend/internal/util/logger"
"strconv"
@@ -70,6 +73,22 @@ func Test_Storage_BasicOperations(t *testing.T) {
}
}
// Setup FTP port
ftpPort := 21
if portStr := config.GetEnv().TestFTPPort; portStr != "" {
if port, err := strconv.Atoi(portStr); err == nil {
ftpPort = port
}
}
// Setup SFTP port
sftpPort := 22
if portStr := config.GetEnv().TestSFTPPort; portStr != "" {
if port, err := strconv.Atoi(portStr); err == nil {
sftpPort = port
}
}
// Run tests
testCases := []struct {
name string
@@ -124,6 +143,44 @@ func Test_Storage_BasicOperations(t *testing.T) {
ContainerName: azuriteContainer.containerNameStr,
},
},
{
name: "FTPStorage",
storage: &ftp_storage.FTPStorage{
StorageID: uuid.New(),
Host: "localhost",
Port: ftpPort,
Username: "testuser",
Password: "testpassword",
UseSSL: false,
Path: "test-files",
},
},
{
name: "SFTPStorage",
storage: &sftp_storage.SFTPStorage{
StorageID: uuid.New(),
Host: "localhost",
Port: sftpPort,
Username: "testuser",
Password: "testpassword",
SkipHostKeyVerify: true,
Path: "upload",
},
},
{
name: "RcloneStorage",
storage: &rclone_storage.RcloneStorage{
StorageID: uuid.New(),
ConfigContent: fmt.Sprintf(`[minio]
type = s3
provider = Other
access_key_id = %s
secret_access_key = %s
endpoint = http://%s
acl = private`, s3Container.accessKey, s3Container.secretKey, s3Container.endpoint),
RemotePath: s3Container.bucketName,
},
},
}
// Add Google Drive storage test only if environment variables are available
@@ -167,6 +224,7 @@ func Test_Storage_BasicOperations(t *testing.T) {
fileID := uuid.New()
err = tc.storage.SaveFile(
context.Background(),
encryptor,
logger.GetLogger(),
fileID,
@@ -189,6 +247,7 @@ func Test_Storage_BasicOperations(t *testing.T) {
fileID := uuid.New()
err = tc.storage.SaveFile(
context.Background(),
encryptor,
logger.GetLogger(),
fileID,
@@ -238,7 +297,7 @@ func setupS3Container(ctx context.Context) (*S3Container, error) {
secretKey := "testpassword"
bucketName := "test-bucket"
region := "us-east-1"
endpoint := fmt.Sprintf("localhost:%s", env.TestMinioPort)
endpoint := fmt.Sprintf("127.0.0.1:%s", env.TestMinioPort)
// Create MinIO client and ensure bucket exists
minioClient, err := minio.New(endpoint, &minio.Options{

View File

@@ -3,19 +3,44 @@ package azure_blob_storage
import (
"bytes"
"context"
"encoding/base64"
"errors"
"fmt"
"io"
"log/slog"
"net"
"net/http"
"postgresus-backend/internal/util/encryption"
"strings"
"time"
"github.com/Azure/azure-sdk-for-go/sdk/azcore"
"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blockblob"
"github.com/google/uuid"
)
const (
azureConnectTimeout = 30 * time.Second
azureResponseTimeout = 30 * time.Second
azureIdleConnTimeout = 90 * time.Second
azureTLSHandshakeTimeout = 30 * time.Second
// Chunk size for block blob uploads - 16MB provides good balance between
// memory usage and upload efficiency. This creates backpressure to pg_dump
// by only reading one chunk at a time and waiting for Azure to confirm receipt.
azureChunkSize = 16 * 1024 * 1024
)
type readSeekCloser struct {
*bytes.Reader
}
func (r *readSeekCloser) Close() error {
return nil
}
type AuthMethod string
const (
@@ -39,27 +64,91 @@ func (s *AzureBlobStorage) TableName() string {
}
func (s *AzureBlobStorage) SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,
file io.Reader,
) error {
select {
case <-ctx.Done():
return fmt.Errorf("upload cancelled before start: %w", ctx.Err())
default:
}
client, err := s.getClient(encryptor)
if err != nil {
return err
}
blobName := s.buildBlobName(fileID.String())
blockBlobClient := client.ServiceClient().
NewContainerClient(s.ContainerName).
NewBlockBlobClient(blobName)
_, err = client.UploadStream(
context.TODO(),
s.ContainerName,
blobName,
file,
nil,
)
var blockIDs []string
blockNumber := 0
buf := make([]byte, azureChunkSize)
for {
select {
case <-ctx.Done():
return fmt.Errorf("upload cancelled: %w", ctx.Err())
default:
}
n, readErr := io.ReadFull(file, buf)
if n == 0 && readErr == io.EOF {
break
}
if readErr != nil && readErr != io.EOF && readErr != io.ErrUnexpectedEOF {
return fmt.Errorf("read error: %w", readErr)
}
blockID := base64.StdEncoding.EncodeToString([]byte(fmt.Sprintf("%06d", blockNumber)))
_, err := blockBlobClient.StageBlock(
ctx,
blockID,
&readSeekCloser{bytes.NewReader(buf[:n])},
nil,
)
if err != nil {
select {
case <-ctx.Done():
return fmt.Errorf("upload cancelled: %w", ctx.Err())
default:
return fmt.Errorf("failed to stage block %d: %w", blockNumber, err)
}
}
blockIDs = append(blockIDs, blockID)
blockNumber++
if readErr == io.EOF || readErr == io.ErrUnexpectedEOF {
break
}
}
if len(blockIDs) == 0 {
_, err = client.UploadStream(
ctx,
s.ContainerName,
blobName,
bytes.NewReader([]byte{}),
nil,
)
if err != nil {
return fmt.Errorf("failed to upload empty blob: %w", err)
}
return nil
}
_, err = blockBlobClient.CommitBlockList(ctx, blockIDs, &blockblob.CommitBlockListOptions{})
if err != nil {
return fmt.Errorf("failed to upload blob to Azure: %w", err)
return fmt.Errorf("failed to commit block list: %w", err)
}
return nil
@@ -253,6 +342,8 @@ func (s *AzureBlobStorage) getClient(encryptor encryption.FieldEncryptor) (*azbl
var client *azblob.Client
var err error
clientOptions := s.buildClientOptions()
switch s.AuthMethod {
case AuthMethodConnectionString:
connectionString, decryptErr := encryptor.Decrypt(s.StorageID, s.ConnectionString)
@@ -260,7 +351,7 @@ func (s *AzureBlobStorage) getClient(encryptor encryption.FieldEncryptor) (*azbl
return nil, fmt.Errorf("failed to decrypt Azure connection string: %w", decryptErr)
}
client, err = azblob.NewClientFromConnectionString(connectionString, nil)
client, err = azblob.NewClientFromConnectionString(connectionString, clientOptions)
if err != nil {
return nil, fmt.Errorf(
"failed to create Azure Blob client from connection string: %w",
@@ -279,7 +370,7 @@ func (s *AzureBlobStorage) getClient(encryptor encryption.FieldEncryptor) (*azbl
return nil, fmt.Errorf("failed to create Azure shared key credential: %w", credErr)
}
client, err = azblob.NewClientWithSharedKeyCredential(accountURL, credential, nil)
client, err = azblob.NewClientWithSharedKeyCredential(accountURL, credential, clientOptions)
if err != nil {
return nil, fmt.Errorf("failed to create Azure Blob client with shared key: %w", err)
}
@@ -290,6 +381,26 @@ func (s *AzureBlobStorage) getClient(encryptor encryption.FieldEncryptor) (*azbl
return client, nil
}
func (s *AzureBlobStorage) buildClientOptions() *azblob.ClientOptions {
transport := &http.Transport{
DialContext: (&net.Dialer{
Timeout: azureConnectTimeout,
}).DialContext,
TLSHandshakeTimeout: azureTLSHandshakeTimeout,
ResponseHeaderTimeout: azureResponseTimeout,
IdleConnTimeout: azureIdleConnTimeout,
}
return &azblob.ClientOptions{
ClientOptions: azcore.ClientOptions{
Transport: &http.Client{Transport: transport},
Retry: policy.RetryOptions{
MaxRetries: 0,
},
},
}
}
func (s *AzureBlobStorage) buildAccountURL() string {
if s.Endpoint != "" {
endpoint := s.Endpoint

View File

@@ -0,0 +1,368 @@
package ftp_storage
import (
"context"
"crypto/tls"
"errors"
"fmt"
"io"
"log/slog"
"postgresus-backend/internal/util/encryption"
"strings"
"time"
"github.com/google/uuid"
"github.com/jlaffaye/ftp"
)
const (
ftpConnectTimeout = 30 * time.Second
ftpTestConnectTimeout = 10 * time.Second
ftpChunkSize = 16 * 1024 * 1024
)
type FTPStorage struct {
StorageID uuid.UUID `json:"storageId" gorm:"primaryKey;type:uuid;column:storage_id"`
Host string `json:"host" gorm:"not null;type:text;column:host"`
Port int `json:"port" gorm:"not null;default:21;column:port"`
Username string `json:"username" gorm:"not null;type:text;column:username"`
Password string `json:"password" gorm:"not null;type:text;column:password"`
Path string `json:"path" gorm:"type:text;column:path"`
UseSSL bool `json:"useSsl" gorm:"not null;default:false;column:use_ssl"`
SkipTLSVerify bool `json:"skipTlsVerify" gorm:"not null;default:false;column:skip_tls_verify"`
}
func (f *FTPStorage) TableName() string {
return "ftp_storages"
}
func (f *FTPStorage) SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,
file io.Reader,
) error {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
logger.Info("Starting to save file to FTP storage", "fileId", fileID.String(), "host", f.Host)
conn, err := f.connect(encryptor, ftpConnectTimeout)
if err != nil {
logger.Error("Failed to connect to FTP", "fileId", fileID.String(), "error", err)
return fmt.Errorf("failed to connect to FTP: %w", err)
}
defer func() {
if quitErr := conn.Quit(); quitErr != nil {
logger.Error(
"Failed to close FTP connection",
"fileId",
fileID.String(),
"error",
quitErr,
)
}
}()
if f.Path != "" {
if err := f.ensureDirectory(conn, f.Path); err != nil {
logger.Error(
"Failed to ensure directory",
"fileId",
fileID.String(),
"path",
f.Path,
"error",
err,
)
return fmt.Errorf("failed to ensure directory: %w", err)
}
}
filePath := f.getFilePath(fileID.String())
logger.Debug("Uploading file to FTP", "fileId", fileID.String(), "filePath", filePath)
ctxReader := &contextReader{ctx: ctx, reader: file}
err = conn.Stor(filePath, ctxReader)
if err != nil {
select {
case <-ctx.Done():
logger.Info("FTP upload cancelled", "fileId", fileID.String())
return ctx.Err()
default:
logger.Error("Failed to upload file to FTP", "fileId", fileID.String(), "error", err)
return fmt.Errorf("failed to upload file to FTP: %w", err)
}
}
logger.Info(
"Successfully saved file to FTP storage",
"fileId",
fileID.String(),
"filePath",
filePath,
)
return nil
}
func (f *FTPStorage) GetFile(
encryptor encryption.FieldEncryptor,
fileID uuid.UUID,
) (io.ReadCloser, error) {
conn, err := f.connect(encryptor, ftpConnectTimeout)
if err != nil {
return nil, fmt.Errorf("failed to connect to FTP: %w", err)
}
filePath := f.getFilePath(fileID.String())
resp, err := conn.Retr(filePath)
if err != nil {
_ = conn.Quit()
return nil, fmt.Errorf("failed to retrieve file from FTP: %w", err)
}
return &ftpFileReader{
response: resp,
conn: conn,
}, nil
}
func (f *FTPStorage) DeleteFile(encryptor encryption.FieldEncryptor, fileID uuid.UUID) error {
conn, err := f.connect(encryptor, ftpConnectTimeout)
if err != nil {
return fmt.Errorf("failed to connect to FTP: %w", err)
}
defer func() {
_ = conn.Quit()
}()
filePath := f.getFilePath(fileID.String())
_, err = conn.FileSize(filePath)
if err != nil {
return nil
}
err = conn.Delete(filePath)
if err != nil {
return fmt.Errorf("failed to delete file from FTP: %w", err)
}
return nil
}
func (f *FTPStorage) Validate(encryptor encryption.FieldEncryptor) error {
if f.Host == "" {
return errors.New("FTP host is required")
}
if f.Username == "" {
return errors.New("FTP username is required")
}
if f.Password == "" {
return errors.New("FTP password is required")
}
if f.Port <= 0 || f.Port > 65535 {
return errors.New("FTP port must be between 1 and 65535")
}
return nil
}
func (f *FTPStorage) TestConnection(encryptor encryption.FieldEncryptor) error {
ctx, cancel := context.WithTimeout(context.Background(), ftpTestConnectTimeout)
defer cancel()
conn, err := f.connectWithContext(ctx, encryptor, ftpTestConnectTimeout)
if err != nil {
return fmt.Errorf("failed to connect to FTP: %w", err)
}
defer func() {
_ = conn.Quit()
}()
if f.Path != "" {
if err := f.ensureDirectory(conn, f.Path); err != nil {
return fmt.Errorf("failed to access or create path '%s': %w", f.Path, err)
}
}
return nil
}
func (f *FTPStorage) HideSensitiveData() {
f.Password = ""
}
func (f *FTPStorage) EncryptSensitiveData(encryptor encryption.FieldEncryptor) error {
if f.Password != "" {
encrypted, err := encryptor.Encrypt(f.StorageID, f.Password)
if err != nil {
return fmt.Errorf("failed to encrypt FTP password: %w", err)
}
f.Password = encrypted
}
return nil
}
func (f *FTPStorage) Update(incoming *FTPStorage) {
f.Host = incoming.Host
f.Port = incoming.Port
f.Username = incoming.Username
f.UseSSL = incoming.UseSSL
f.SkipTLSVerify = incoming.SkipTLSVerify
f.Path = incoming.Path
if incoming.Password != "" {
f.Password = incoming.Password
}
}
func (f *FTPStorage) connect(
encryptor encryption.FieldEncryptor,
timeout time.Duration,
) (*ftp.ServerConn, error) {
return f.connectWithContext(context.Background(), encryptor, timeout)
}
func (f *FTPStorage) connectWithContext(
ctx context.Context,
encryptor encryption.FieldEncryptor,
timeout time.Duration,
) (*ftp.ServerConn, error) {
password, err := encryptor.Decrypt(f.StorageID, f.Password)
if err != nil {
return nil, fmt.Errorf("failed to decrypt FTP password: %w", err)
}
address := fmt.Sprintf("%s:%d", f.Host, f.Port)
dialCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
var conn *ftp.ServerConn
if f.UseSSL {
tlsConfig := &tls.Config{
ServerName: f.Host,
InsecureSkipVerify: f.SkipTLSVerify,
}
conn, err = ftp.Dial(address,
ftp.DialWithContext(dialCtx),
ftp.DialWithExplicitTLS(tlsConfig),
)
} else {
conn, err = ftp.Dial(address, ftp.DialWithContext(dialCtx))
}
if err != nil {
return nil, fmt.Errorf("failed to dial FTP server: %w", err)
}
err = conn.Login(f.Username, password)
if err != nil {
_ = conn.Quit()
return nil, fmt.Errorf("failed to login to FTP server: %w", err)
}
return conn, nil
}
func (f *FTPStorage) ensureDirectory(conn *ftp.ServerConn, path string) error {
path = strings.TrimPrefix(path, "/")
path = strings.TrimSuffix(path, "/")
if path == "" {
return nil
}
parts := strings.Split(path, "/")
currentPath := ""
for _, part := range parts {
if part == "" || part == "." {
continue
}
if currentPath == "" {
currentPath = part
} else {
currentPath = currentPath + "/" + part
}
err := conn.ChangeDir(currentPath)
if err != nil {
err = conn.MakeDir(currentPath)
if err != nil {
return fmt.Errorf("failed to create directory '%s': %w", currentPath, err)
}
}
err = conn.ChangeDirToParent()
if err != nil {
return fmt.Errorf("failed to change to parent directory: %w", err)
}
}
return nil
}
func (f *FTPStorage) getFilePath(filename string) string {
if f.Path == "" {
return filename
}
path := strings.TrimPrefix(f.Path, "/")
path = strings.TrimSuffix(path, "/")
return path + "/" + filename
}
type ftpFileReader struct {
response *ftp.Response
conn *ftp.ServerConn
}
func (r *ftpFileReader) Read(p []byte) (n int, err error) {
return r.response.Read(p)
}
func (r *ftpFileReader) Close() error {
var errs []error
if r.response != nil {
if err := r.response.Close(); err != nil {
errs = append(errs, fmt.Errorf("failed to close response: %w", err))
}
}
if r.conn != nil {
if err := r.conn.Quit(); err != nil {
errs = append(errs, fmt.Errorf("failed to close connection: %w", err))
}
}
if len(errs) > 0 {
return errs[0]
}
return nil
}
type contextReader struct {
ctx context.Context
reader io.Reader
}
func (r *contextReader) Read(p []byte) (n int, err error) {
select {
case <-r.ctx.Done():
return 0, r.ctx.Err()
default:
return r.reader.Read(p)
}
}

View File

@@ -7,6 +7,8 @@ import (
"fmt"
"io"
"log/slog"
"net"
"net/http"
"postgresus-backend/internal/util/encryption"
"strings"
"time"
@@ -16,9 +18,22 @@ import (
"golang.org/x/oauth2/google"
drive "google.golang.org/api/drive/v3"
"google.golang.org/api/googleapi"
"google.golang.org/api/option"
)
const (
gdConnectTimeout = 30 * time.Second
gdResponseTimeout = 30 * time.Second
gdIdleConnTimeout = 90 * time.Second
gdTLSHandshakeTimeout = 30 * time.Second
// Chunk size for Google Drive resumable uploads - 16MB provides good balance
// between memory usage and upload efficiency. Google Drive requires chunks
// to be multiples of 256KB for resumable uploads.
gdChunkSize = 16 * 1024 * 1024
)
type GoogleDriveStorage struct {
StorageID uuid.UUID `json:"storageId" gorm:"primaryKey;type:uuid;column:storage_id"`
ClientID string `json:"clientId" gorm:"not null;type:text;column:client_id"`
@@ -31,31 +46,44 @@ func (s *GoogleDriveStorage) TableName() string {
}
func (s *GoogleDriveStorage) SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,
file io.Reader,
) error {
return s.withRetryOnAuth(encryptor, func(driveService *drive.Service) error {
ctx := context.Background()
return s.withRetryOnAuth(ctx, encryptor, func(driveService *drive.Service) error {
filename := fileID.String()
// Ensure the postgresus_backups folder exists
folderID, err := s.ensureBackupsFolderExists(ctx, driveService)
if err != nil {
return fmt.Errorf("failed to create/find backups folder: %w", err)
}
// Delete any previous copy so we keep at most one object per logical file.
_ = s.deleteByName(ctx, driveService, filename, folderID) // ignore "not found"
_ = s.deleteByName(ctx, driveService, filename, folderID)
fileMeta := &drive.File{
Name: filename,
Parents: []string{folderID},
}
_, err = driveService.Files.Create(fileMeta).Media(file).Context(ctx).Do()
backpressureReader := &backpressureReader{
reader: file,
ctx: ctx,
chunkSize: gdChunkSize,
buf: make([]byte, gdChunkSize),
}
_, err = driveService.Files.Create(fileMeta).
Media(backpressureReader, googleapi.ChunkSize(gdChunkSize)).
Context(ctx).
Do()
if err != nil {
select {
case <-ctx.Done():
return fmt.Errorf("upload cancelled: %w", ctx.Err())
default:
}
return fmt.Errorf("failed to upload file to Google Drive: %w", err)
}
@@ -70,30 +98,85 @@ func (s *GoogleDriveStorage) SaveFile(
})
}
type backpressureReader struct {
reader io.Reader
ctx context.Context
chunkSize int
buf []byte
bufStart int
bufEnd int
totalBytes int64
chunkCount int
}
func (r *backpressureReader) Read(p []byte) (n int, err error) {
select {
case <-r.ctx.Done():
return 0, r.ctx.Err()
default:
}
if r.bufStart >= r.bufEnd {
r.chunkCount++
bytesRead, readErr := io.ReadFull(r.reader, r.buf)
if bytesRead > 0 {
r.bufStart = 0
r.bufEnd = bytesRead
}
if readErr != nil && readErr != io.EOF && readErr != io.ErrUnexpectedEOF {
return 0, readErr
}
if bytesRead == 0 && readErr == io.EOF {
return 0, io.EOF
}
}
n = copy(p, r.buf[r.bufStart:r.bufEnd])
r.bufStart += n
r.totalBytes += int64(n)
if r.bufStart >= r.bufEnd {
select {
case <-r.ctx.Done():
return n, r.ctx.Err()
default:
}
}
return n, nil
}
func (s *GoogleDriveStorage) GetFile(
encryptor encryption.FieldEncryptor,
fileID uuid.UUID,
) (io.ReadCloser, error) {
var result io.ReadCloser
err := s.withRetryOnAuth(encryptor, func(driveService *drive.Service) error {
folderID, err := s.findBackupsFolder(driveService)
if err != nil {
return fmt.Errorf("failed to find backups folder: %w", err)
}
err := s.withRetryOnAuth(
context.Background(),
encryptor,
func(driveService *drive.Service) error {
folderID, err := s.findBackupsFolder(driveService)
if err != nil {
return fmt.Errorf("failed to find backups folder: %w", err)
}
fileIDGoogle, err := s.lookupFileID(driveService, fileID.String(), folderID)
if err != nil {
return err
}
fileIDGoogle, err := s.lookupFileID(driveService, fileID.String(), folderID)
if err != nil {
return err
}
resp, err := driveService.Files.Get(fileIDGoogle).Download()
if err != nil {
return fmt.Errorf("failed to download file from Google Drive: %w", err)
}
resp, err := driveService.Files.Get(fileIDGoogle).Download()
if err != nil {
return fmt.Errorf("failed to download file from Google Drive: %w", err)
}
result = resp.Body
return nil
})
result = resp.Body
return nil
},
)
return result, err
}
@@ -102,8 +185,8 @@ func (s *GoogleDriveStorage) DeleteFile(
encryptor encryption.FieldEncryptor,
fileID uuid.UUID,
) error {
return s.withRetryOnAuth(encryptor, func(driveService *drive.Service) error {
ctx := context.Background()
ctx := context.Background()
return s.withRetryOnAuth(ctx, encryptor, func(driveService *drive.Service) error {
folderID, err := s.findBackupsFolder(driveService)
if err != nil {
return fmt.Errorf("failed to find backups folder: %w", err)
@@ -142,8 +225,8 @@ func (s *GoogleDriveStorage) Validate(encryptor encryption.FieldEncryptor) error
}
func (s *GoogleDriveStorage) TestConnection(encryptor encryption.FieldEncryptor) error {
return s.withRetryOnAuth(encryptor, func(driveService *drive.Service) error {
ctx := context.Background()
ctx := context.Background()
return s.withRetryOnAuth(ctx, encryptor, func(driveService *drive.Service) error {
testFilename := "test-connection-" + uuid.New().String()
testData := []byte("test")
@@ -243,9 +326,16 @@ func (s *GoogleDriveStorage) Update(incoming *GoogleDriveStorage) {
// withRetryOnAuth executes the provided function with retry logic for authentication errors
func (s *GoogleDriveStorage) withRetryOnAuth(
ctx context.Context,
encryptor encryption.FieldEncryptor,
fn func(*drive.Service) error,
) error {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
driveService, err := s.getDriveService(encryptor)
if err != nil {
return err
@@ -253,6 +343,12 @@ func (s *GoogleDriveStorage) withRetryOnAuth(
err = fn(driveService)
if err != nil && s.isAuthError(err) {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
// Try to refresh token and retry once
fmt.Printf("Google Drive auth error detected, attempting token refresh: %v\n", err)
@@ -422,7 +518,6 @@ func (s *GoogleDriveStorage) getDriveService(
return nil, err
}
// Decrypt credentials before use
clientSecret, err := encryptor.Decrypt(s.StorageID, s.ClientSecret)
if err != nil {
return nil, fmt.Errorf("failed to decrypt Google Drive client secret: %w", err)
@@ -449,16 +544,16 @@ func (s *GoogleDriveStorage) getDriveService(
tokenSource := cfg.TokenSource(ctx, &token)
// Force token validation to ensure we're using the current token
currentToken, err := tokenSource.Token()
if err != nil {
return nil, fmt.Errorf("failed to get current token: %w", err)
}
// Create a new token source with the validated token
validatedTokenSource := oauth2.StaticTokenSource(currentToken)
driveService, err := drive.NewService(ctx, option.WithTokenSource(validatedTokenSource))
httpClient := s.buildHTTPClient(validatedTokenSource)
driveService, err := drive.NewService(ctx, option.WithHTTPClient(httpClient))
if err != nil {
return nil, fmt.Errorf("unable to create Drive client: %w", err)
}
@@ -466,6 +561,24 @@ func (s *GoogleDriveStorage) getDriveService(
return driveService, nil
}
func (s *GoogleDriveStorage) buildHTTPClient(tokenSource oauth2.TokenSource) *http.Client {
transport := &http.Transport{
DialContext: (&net.Dialer{
Timeout: gdConnectTimeout,
}).DialContext,
TLSHandshakeTimeout: gdTLSHandshakeTimeout,
ResponseHeaderTimeout: gdResponseTimeout,
IdleConnTimeout: gdIdleConnTimeout,
}
return &http.Client{
Transport: &oauth2.Transport{
Source: tokenSource,
Base: transport,
},
}
}
func (s *GoogleDriveStorage) lookupFileID(
driveService *drive.Service,
name string,

View File

@@ -1,6 +1,7 @@
package local_storage
import (
"context"
"fmt"
"io"
"log/slog"
@@ -13,6 +14,13 @@ import (
"github.com/google/uuid"
)
const (
// Chunk size for local storage writes - 8MB per buffer with double-buffering
// allows overlapped I/O while keeping total memory under 32MB.
// Two 8MB buffers = 16MB for local storage, plus 8MB for pg_dump buffer = ~25MB total.
localChunkSize = 8 * 1024 * 1024
)
// LocalStorage uses ./postgresus_local_backups folder as a
// directory for backups and ./postgresus_local_temp folder as a
// directory for temp files
@@ -25,11 +33,18 @@ func (l *LocalStorage) TableName() string {
}
func (l *LocalStorage) SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,
file io.Reader,
) error {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
logger.Info("Starting to save file to local storage", "fileId", fileID.String())
err := files_utils.EnsureDirectories([]string{
@@ -60,7 +75,7 @@ func (l *LocalStorage) SaveFile(
}()
logger.Debug("Copying file data to temp file", "fileId", fileID.String())
_, err = io.Copy(tempFile, file)
_, err = copyWithContext(ctx, tempFile, file)
if err != nil {
logger.Error("Failed to write to temp file", "fileId", fileID.String(), "error", err)
return fmt.Errorf("failed to write to temp file: %w", err)
@@ -175,3 +190,35 @@ func (l *LocalStorage) EncryptSensitiveData(encryptor encryption.FieldEncryptor)
func (l *LocalStorage) Update(incoming *LocalStorage) {
}
func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, error) {
buf := make([]byte, localChunkSize)
var written int64
for {
select {
case <-ctx.Done():
return written, ctx.Err()
default:
}
nr, readErr := src.Read(buf)
if nr > 0 {
nw, writeErr := dst.Write(buf[:nr])
written += int64(nw)
if writeErr != nil {
return written, writeErr
}
if nr != nw {
return written, io.ErrShortWrite
}
}
if readErr == io.EOF {
return written, nil
}
if readErr != nil {
return written, readErr
}
}
}

View File

@@ -1,6 +1,7 @@
package nas_storage
import (
"context"
"crypto/tls"
"errors"
"fmt"
@@ -16,6 +17,13 @@ import (
"github.com/hirochachacha/go-smb2"
)
const (
// Chunk size for NAS uploads - 16MB provides good balance between
// memory usage and upload efficiency. This creates backpressure to pg_dump
// by only reading one chunk at a time and waiting for NAS to confirm receipt.
nasChunkSize = 16 * 1024 * 1024
)
type NASStorage struct {
StorageID uuid.UUID `json:"storageId" gorm:"primaryKey;type:uuid;column:storage_id"`
Host string `json:"host" gorm:"not null;type:text;column:host"`
@@ -33,14 +41,21 @@ func (n *NASStorage) TableName() string {
}
func (n *NASStorage) SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,
file io.Reader,
) error {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
logger.Info("Starting to save file to NAS storage", "fileId", fileID.String(), "host", n.Host)
session, err := n.createSession(encryptor)
session, err := n.createSessionWithContext(ctx, encryptor)
if err != nil {
logger.Error("Failed to create NAS session", "fileId", fileID.String(), "error", err)
return fmt.Errorf("failed to create NAS session: %w", err)
@@ -121,7 +136,7 @@ func (n *NASStorage) SaveFile(
}()
logger.Debug("Copying file data to NAS", "fileId", fileID.String())
_, err = io.Copy(nasFile, file)
_, err = copyWithContext(ctx, nasFile, file)
if err != nil {
logger.Error("Failed to write file to NAS", "fileId", fileID.String(), "error", err)
return fmt.Errorf("failed to write file to NAS: %w", err)
@@ -290,20 +305,24 @@ func (n *NASStorage) Update(incoming *NASStorage) {
}
func (n *NASStorage) createSession(encryptor encryption.FieldEncryptor) (*smb2.Session, error) {
// Create connection with timeout
conn, err := n.createConnection()
return n.createSessionWithContext(context.Background(), encryptor)
}
func (n *NASStorage) createSessionWithContext(
ctx context.Context,
encryptor encryption.FieldEncryptor,
) (*smb2.Session, error) {
conn, err := n.createConnectionWithContext(ctx)
if err != nil {
return nil, err
}
// Decrypt password before use
password, err := encryptor.Decrypt(n.StorageID, n.Password)
if err != nil {
_ = conn.Close()
return nil, fmt.Errorf("failed to decrypt NAS password: %w", err)
}
// Create SMB2 dialer
d := &smb2.Dialer{
Initiator: &smb2.NTLMInitiator{
User: n.Username,
@@ -312,7 +331,6 @@ func (n *NASStorage) createSession(encryptor encryption.FieldEncryptor) (*smb2.S
},
}
// Create session
session, err := d.Dial(conn)
if err != nil {
_ = conn.Close()
@@ -322,34 +340,30 @@ func (n *NASStorage) createSession(encryptor encryption.FieldEncryptor) (*smb2.S
return session, nil
}
func (n *NASStorage) createConnection() (net.Conn, error) {
func (n *NASStorage) createConnectionWithContext(ctx context.Context) (net.Conn, error) {
address := net.JoinHostPort(n.Host, fmt.Sprintf("%d", n.Port))
// Create connection with timeout
dialer := &net.Dialer{
Timeout: 10 * time.Second,
Timeout: 30 * time.Second,
}
if n.UseSSL {
// Use TLS connection
tlsConfig := &tls.Config{
ServerName: n.Host,
InsecureSkipVerify: false, // Change to true if you want to skip cert verification
InsecureSkipVerify: false,
}
conn, err := tls.DialWithDialer(dialer, "tcp", address, tlsConfig)
if err != nil {
return nil, fmt.Errorf("failed to create SSL connection to %s: %w", address, err)
}
return conn, nil
} else {
// Use regular TCP connection
conn, err := dialer.Dial("tcp", address)
if err != nil {
return nil, fmt.Errorf("failed to create connection to %s: %w", address, err)
}
return conn, nil
}
conn, err := dialer.DialContext(ctx, "tcp", address)
if err != nil {
return nil, fmt.Errorf("failed to create connection to %s: %w", address, err)
}
return conn, nil
}
func (n *NASStorage) ensureDirectory(fs *smb2.Share, path string) error {
@@ -444,3 +458,71 @@ func (r *nasFileReader) Close() error {
return nil
}
type writeResult struct {
bytesWritten int
writeErr error
}
func copyWithContext(ctx context.Context, dst io.Writer, src io.Reader) (int64, error) {
buf := make([]byte, nasChunkSize)
var written int64
for {
select {
case <-ctx.Done():
return written, ctx.Err()
default:
}
nr, readErr := io.ReadFull(src, buf)
if nr == 0 && readErr == io.EOF {
break
}
if readErr != nil && readErr != io.EOF && readErr != io.ErrUnexpectedEOF {
return written, readErr
}
writeResultCh := make(chan writeResult, 1)
go func() {
nw, writeErr := dst.Write(buf[0:nr])
writeResultCh <- writeResult{nw, writeErr}
}()
var nw int
var writeErr error
select {
case <-ctx.Done():
return written, ctx.Err()
case result := <-writeResultCh:
nw = result.bytesWritten
writeErr = result.writeErr
}
if nw < 0 || nr < nw {
nw = 0
if writeErr == nil {
writeErr = errors.New("invalid write result")
}
}
if writeErr != nil {
return written, writeErr
}
if nr != nw {
return written, io.ErrShortWrite
}
written += int64(nw)
if readErr == io.EOF || readErr == io.ErrUnexpectedEOF {
break
}
}
return written, nil
}

View File

@@ -0,0 +1,293 @@
package rclone_storage
import (
"bufio"
"context"
"errors"
"fmt"
"io"
"log/slog"
"postgresus-backend/internal/util/encryption"
"strings"
"sync"
"time"
"github.com/google/uuid"
"github.com/rclone/rclone/fs"
"github.com/rclone/rclone/fs/config"
"github.com/rclone/rclone/fs/operations"
_ "github.com/rclone/rclone/backend/all"
)
const (
rcloneOperationTimeout = 30 * time.Second
)
var rcloneConfigMu sync.Mutex
type RcloneStorage struct {
StorageID uuid.UUID `json:"storageId" gorm:"primaryKey;type:uuid;column:storage_id"`
ConfigContent string `json:"configContent" gorm:"not null;type:text;column:config_content"`
RemotePath string `json:"remotePath" gorm:"type:text;column:remote_path"`
}
func (r *RcloneStorage) TableName() string {
return "rclone_storages"
}
func (r *RcloneStorage) SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,
file io.Reader,
) error {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
logger.Info("Starting to save file to rclone storage", "fileId", fileID.String())
remoteFs, err := r.getFs(ctx, encryptor)
if err != nil {
logger.Error("Failed to create rclone filesystem", "fileId", fileID.String(), "error", err)
return fmt.Errorf("failed to create rclone filesystem: %w", err)
}
filePath := r.getFilePath(fileID.String())
logger.Debug("Uploading file via rclone", "fileId", fileID.String(), "filePath", filePath)
_, err = operations.Rcat(ctx, remoteFs, filePath, io.NopCloser(file), time.Now().UTC(), nil)
if err != nil {
select {
case <-ctx.Done():
logger.Info("Rclone upload cancelled", "fileId", fileID.String())
return ctx.Err()
default:
logger.Error(
"Failed to upload file via rclone",
"fileId",
fileID.String(),
"error",
err,
)
return fmt.Errorf("failed to upload file via rclone: %w", err)
}
}
logger.Info(
"Successfully saved file to rclone storage",
"fileId",
fileID.String(),
"filePath",
filePath,
)
return nil
}
func (r *RcloneStorage) GetFile(
encryptor encryption.FieldEncryptor,
fileID uuid.UUID,
) (io.ReadCloser, error) {
ctx := context.Background()
remoteFs, err := r.getFs(ctx, encryptor)
if err != nil {
return nil, fmt.Errorf("failed to create rclone filesystem: %w", err)
}
filePath := r.getFilePath(fileID.String())
obj, err := remoteFs.NewObject(ctx, filePath)
if err != nil {
return nil, fmt.Errorf("failed to get object from rclone: %w", err)
}
reader, err := obj.Open(ctx)
if err != nil {
return nil, fmt.Errorf("failed to open object from rclone: %w", err)
}
return reader, nil
}
func (r *RcloneStorage) DeleteFile(encryptor encryption.FieldEncryptor, fileID uuid.UUID) error {
ctx := context.Background()
remoteFs, err := r.getFs(ctx, encryptor)
if err != nil {
return fmt.Errorf("failed to create rclone filesystem: %w", err)
}
filePath := r.getFilePath(fileID.String())
obj, err := remoteFs.NewObject(ctx, filePath)
if err != nil {
return nil
}
err = obj.Remove(ctx)
if err != nil {
return fmt.Errorf("failed to delete file from rclone: %w", err)
}
return nil
}
func (r *RcloneStorage) Validate(encryptor encryption.FieldEncryptor) error {
if r.ConfigContent == "" {
return errors.New("rclone config content is required")
}
return nil
}
func (r *RcloneStorage) TestConnection(encryptor encryption.FieldEncryptor) error {
ctx, cancel := context.WithTimeout(context.Background(), rcloneOperationTimeout)
defer cancel()
remoteFs, err := r.getFs(ctx, encryptor)
if err != nil {
return fmt.Errorf("failed to create rclone filesystem: %w", err)
}
testFileID := uuid.New().String() + "-test"
testFilePath := r.getFilePath(testFileID)
testData := strings.NewReader("test connection")
_, err = operations.Rcat(
ctx,
remoteFs,
testFilePath,
io.NopCloser(testData),
time.Now().UTC(),
nil,
)
if err != nil {
return fmt.Errorf("failed to upload test file via rclone: %w", err)
}
obj, err := remoteFs.NewObject(ctx, testFilePath)
if err != nil {
return fmt.Errorf("failed to get test file from rclone: %w", err)
}
err = obj.Remove(ctx)
if err != nil {
return fmt.Errorf("failed to delete test file from rclone: %w", err)
}
return nil
}
func (r *RcloneStorage) HideSensitiveData() {
r.ConfigContent = ""
}
func (r *RcloneStorage) EncryptSensitiveData(encryptor encryption.FieldEncryptor) error {
if r.ConfigContent != "" {
encrypted, err := encryptor.Encrypt(r.StorageID, r.ConfigContent)
if err != nil {
return fmt.Errorf("failed to encrypt rclone config content: %w", err)
}
r.ConfigContent = encrypted
}
return nil
}
func (r *RcloneStorage) Update(incoming *RcloneStorage) {
r.RemotePath = incoming.RemotePath
if incoming.ConfigContent != "" {
r.ConfigContent = incoming.ConfigContent
}
}
func (r *RcloneStorage) getFs(
ctx context.Context,
encryptor encryption.FieldEncryptor,
) (fs.Fs, error) {
configContent, err := encryptor.Decrypt(r.StorageID, r.ConfigContent)
if err != nil {
return nil, fmt.Errorf("failed to decrypt rclone config content: %w", err)
}
rcloneConfigMu.Lock()
defer rcloneConfigMu.Unlock()
parsedConfig, err := parseConfigContent(configContent)
if err != nil {
return nil, fmt.Errorf("failed to parse rclone config: %w", err)
}
if len(parsedConfig) == 0 {
return nil, errors.New("rclone config must contain at least one remote section")
}
var remoteName string
for section, values := range parsedConfig {
remoteName = section
for key, value := range values {
config.FileSetValue(section, key, value)
}
}
remotePath := remoteName + ":"
if r.RemotePath != "" {
remotePath = remoteName + ":" + strings.TrimPrefix(r.RemotePath, "/")
}
remoteFs, err := fs.NewFs(ctx, remotePath)
if err != nil {
return nil, fmt.Errorf(
"failed to create rclone filesystem for remote '%s': %w",
remoteName,
err,
)
}
return remoteFs, nil
}
func (r *RcloneStorage) getFilePath(filename string) string {
return filename
}
func parseConfigContent(content string) (map[string]map[string]string, error) {
sections := make(map[string]map[string]string)
var currentSection string
scanner := bufio.NewScanner(strings.NewReader(content))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" || strings.HasPrefix(line, "#") || strings.HasPrefix(line, ";") {
continue
}
if strings.HasPrefix(line, "[") && strings.HasSuffix(line, "]") {
currentSection = strings.TrimPrefix(strings.TrimSuffix(line, "]"), "[")
if sections[currentSection] == nil {
sections[currentSection] = make(map[string]string)
}
continue
}
if currentSection != "" && strings.Contains(line, "=") {
parts := strings.SplitN(line, "=", 2)
key := strings.TrimSpace(parts[0])
value := ""
if len(parts) > 1 {
value = strings.TrimSpace(parts[1])
}
sections[currentSection][key] = value
}
}
return sections, scanner.Err()
}

View File

@@ -3,10 +3,13 @@ package s3_storage
import (
"bytes"
"context"
"crypto/tls"
"errors"
"fmt"
"io"
"log/slog"
"net"
"net/http"
"postgresus-backend/internal/util/encryption"
"strings"
"time"
@@ -16,6 +19,18 @@ import (
"github.com/minio/minio-go/v7/pkg/credentials"
)
const (
s3ConnectTimeout = 30 * time.Second
s3ResponseTimeout = 30 * time.Second
s3IdleConnTimeout = 90 * time.Second
s3TLSHandshakeTimeout = 30 * time.Second
// Chunk size for multipart uploads - 16MB provides good balance between
// memory usage and upload efficiency. This creates backpressure to pg_dump
// by only reading one chunk at a time and waiting for S3 to confirm receipt.
multipartChunkSize = 16 * 1024 * 1024
)
type S3Storage struct {
StorageID uuid.UUID `json:"storageId" gorm:"primaryKey;type:uuid;column:storage_id"`
S3Bucket string `json:"s3Bucket" gorm:"not null;type:text;column:s3_bucket"`
@@ -26,6 +41,7 @@ type S3Storage struct {
S3Prefix string `json:"s3Prefix" gorm:"type:text;column:s3_prefix"`
S3UseVirtualHostedStyle bool `json:"s3UseVirtualHostedStyle" gorm:"default:false;column:s3_use_virtual_hosted_style"`
SkipTLSVerify bool `json:"skipTLSVerify" gorm:"default:false;column:skip_tls_verify"`
}
func (s *S3Storage) TableName() string {
@@ -33,29 +49,123 @@ func (s *S3Storage) TableName() string {
}
func (s *S3Storage) SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,
file io.Reader,
) error {
client, err := s.getClient(encryptor)
select {
case <-ctx.Done():
return fmt.Errorf("upload cancelled before start: %w", ctx.Err())
default:
}
coreClient, err := s.getCoreClient(encryptor)
if err != nil {
return err
}
objectKey := s.buildObjectKey(fileID.String())
// Upload the file using MinIO client with streaming (size = -1 for unknown size)
_, err = client.PutObject(
context.TODO(),
uploadID, err := coreClient.NewMultipartUpload(
ctx,
s.S3Bucket,
objectKey,
file,
-1,
minio.PutObjectOptions{},
)
if err != nil {
return fmt.Errorf("failed to upload file to S3: %w", err)
return fmt.Errorf("failed to initiate multipart upload: %w", err)
}
var parts []minio.CompletePart
partNumber := 1
buf := make([]byte, multipartChunkSize)
for {
select {
case <-ctx.Done():
_ = coreClient.AbortMultipartUpload(ctx, s.S3Bucket, objectKey, uploadID)
return fmt.Errorf("upload cancelled: %w", ctx.Err())
default:
}
n, readErr := io.ReadFull(file, buf)
if n == 0 && readErr == io.EOF {
break
}
if readErr != nil && readErr != io.EOF && readErr != io.ErrUnexpectedEOF {
_ = coreClient.AbortMultipartUpload(ctx, s.S3Bucket, objectKey, uploadID)
return fmt.Errorf("read error: %w", readErr)
}
part, err := coreClient.PutObjectPart(
ctx,
s.S3Bucket,
objectKey,
uploadID,
partNumber,
bytes.NewReader(buf[:n]),
int64(n),
minio.PutObjectPartOptions{},
)
if err != nil {
_ = coreClient.AbortMultipartUpload(ctx, s.S3Bucket, objectKey, uploadID)
select {
case <-ctx.Done():
return fmt.Errorf("upload cancelled: %w", ctx.Err())
default:
return fmt.Errorf("failed to upload part %d: %w", partNumber, err)
}
}
parts = append(parts, minio.CompletePart{
PartNumber: partNumber,
ETag: part.ETag,
})
partNumber++
if readErr == io.EOF || readErr == io.ErrUnexpectedEOF {
break
}
}
if len(parts) == 0 {
_ = coreClient.AbortMultipartUpload(ctx, s.S3Bucket, objectKey, uploadID)
client, err := s.getClient(encryptor)
if err != nil {
return err
}
_, err = client.PutObject(
ctx,
s.S3Bucket,
objectKey,
bytes.NewReader([]byte{}),
0,
minio.PutObjectOptions{},
)
if err != nil {
return fmt.Errorf("failed to upload empty file: %w", err)
}
return nil
}
_, err = coreClient.CompleteMultipartUpload(
ctx,
s.S3Bucket,
objectKey,
uploadID,
parts,
minio.PutObjectOptions{},
)
if err != nil {
_ = coreClient.AbortMultipartUpload(ctx, s.S3Bucket, objectKey, uploadID)
return fmt.Errorf("failed to complete multipart upload: %w", err)
}
return nil
@@ -223,6 +333,7 @@ func (s *S3Storage) Update(incoming *S3Storage) {
s.S3Region = incoming.S3Region
s.S3Endpoint = incoming.S3Endpoint
s.S3UseVirtualHostedStyle = incoming.S3UseVirtualHostedStyle
s.SkipTLSVerify = incoming.SkipTLSVerify
if incoming.S3AccessKey != "" {
s.S3AccessKey = incoming.S3AccessKey
@@ -252,8 +363,54 @@ func (s *S3Storage) buildObjectKey(fileName string) string {
}
func (s *S3Storage) getClient(encryptor encryption.FieldEncryptor) (*minio.Client, error) {
endpoint := s.S3Endpoint
useSSL := true
endpoint, useSSL, accessKey, secretKey, bucketLookup, transport, err := s.getClientParams(
encryptor,
)
if err != nil {
return nil, err
}
minioClient, err := minio.New(endpoint, &minio.Options{
Creds: credentials.NewStaticV4(accessKey, secretKey, ""),
Secure: useSSL,
Region: s.S3Region,
BucketLookup: bucketLookup,
Transport: transport,
})
if err != nil {
return nil, fmt.Errorf("failed to initialize MinIO client: %w", err)
}
return minioClient, nil
}
func (s *S3Storage) getCoreClient(encryptor encryption.FieldEncryptor) (*minio.Core, error) {
endpoint, useSSL, accessKey, secretKey, bucketLookup, transport, err := s.getClientParams(
encryptor,
)
if err != nil {
return nil, err
}
coreClient, err := minio.NewCore(endpoint, &minio.Options{
Creds: credentials.NewStaticV4(accessKey, secretKey, ""),
Secure: useSSL,
Region: s.S3Region,
BucketLookup: bucketLookup,
Transport: transport,
})
if err != nil {
return nil, fmt.Errorf("failed to initialize MinIO Core client: %w", err)
}
return coreClient, nil
}
func (s *S3Storage) getClientParams(
encryptor encryption.FieldEncryptor,
) (endpoint string, useSSL bool, accessKey string, secretKey string, bucketLookup minio.BucketLookupType, transport *http.Transport, err error) {
endpoint = s.S3Endpoint
useSSL = true
if strings.HasPrefix(endpoint, "http://") {
useSSL = false
@@ -262,38 +419,36 @@ func (s *S3Storage) getClient(encryptor encryption.FieldEncryptor) (*minio.Clien
endpoint = strings.TrimPrefix(endpoint, "https://")
}
// If no endpoint is provided, use the AWS S3 endpoint for the region
if endpoint == "" {
endpoint = fmt.Sprintf("s3.%s.amazonaws.com", s.S3Region)
}
// Decrypt credentials before use
accessKey, err := encryptor.Decrypt(s.StorageID, s.S3AccessKey)
accessKey, err = encryptor.Decrypt(s.StorageID, s.S3AccessKey)
if err != nil {
return nil, fmt.Errorf("failed to decrypt S3 access key: %w", err)
return "", false, "", "", 0, nil, fmt.Errorf("failed to decrypt S3 access key: %w", err)
}
secretKey, err := encryptor.Decrypt(s.StorageID, s.S3SecretKey)
secretKey, err = encryptor.Decrypt(s.StorageID, s.S3SecretKey)
if err != nil {
return nil, fmt.Errorf("failed to decrypt S3 secret key: %w", err)
return "", false, "", "", 0, nil, fmt.Errorf("failed to decrypt S3 secret key: %w", err)
}
// Configure bucket lookup strategy
bucketLookup := minio.BucketLookupAuto
bucketLookup = minio.BucketLookupAuto
if s.S3UseVirtualHostedStyle {
bucketLookup = minio.BucketLookupDNS
}
// Initialize the MinIO client
minioClient, err := minio.New(endpoint, &minio.Options{
Creds: credentials.NewStaticV4(accessKey, secretKey, ""),
Secure: useSSL,
Region: s.S3Region,
BucketLookup: bucketLookup,
})
if err != nil {
return nil, fmt.Errorf("failed to initialize MinIO client: %w", err)
transport = &http.Transport{
DialContext: (&net.Dialer{
Timeout: s3ConnectTimeout,
}).DialContext,
TLSHandshakeTimeout: s3TLSHandshakeTimeout,
ResponseHeaderTimeout: s3ResponseTimeout,
IdleConnTimeout: s3IdleConnTimeout,
TLSClientConfig: &tls.Config{
InsecureSkipVerify: s.SkipTLSVerify,
},
}
return minioClient, nil
return endpoint, useSSL, accessKey, secretKey, bucketLookup, transport, nil
}

View File

@@ -0,0 +1,430 @@
package sftp_storage
import (
"context"
"errors"
"fmt"
"io"
"log/slog"
"net"
"postgresus-backend/internal/util/encryption"
"strings"
"time"
"github.com/google/uuid"
"github.com/pkg/sftp"
"golang.org/x/crypto/ssh"
)
const (
sftpConnectTimeout = 30 * time.Second
sftpTestConnectTimeout = 10 * time.Second
)
type SFTPStorage struct {
StorageID uuid.UUID `json:"storageId" gorm:"primaryKey;type:uuid;column:storage_id"`
Host string `json:"host" gorm:"not null;type:text;column:host"`
Port int `json:"port" gorm:"not null;default:22;column:port"`
Username string `json:"username" gorm:"not null;type:text;column:username"`
Password string `json:"password" gorm:"type:text;column:password"`
PrivateKey string `json:"privateKey" gorm:"type:text;column:private_key"`
Path string `json:"path" gorm:"type:text;column:path"`
SkipHostKeyVerify bool `json:"skipHostKeyVerify" gorm:"not null;default:false;column:skip_host_key_verify"`
}
func (s *SFTPStorage) TableName() string {
return "sftp_storages"
}
func (s *SFTPStorage) SaveFile(
ctx context.Context,
encryptor encryption.FieldEncryptor,
logger *slog.Logger,
fileID uuid.UUID,
file io.Reader,
) error {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
logger.Info("Starting to save file to SFTP storage", "fileId", fileID.String(), "host", s.Host)
client, sshConn, err := s.connect(encryptor, sftpConnectTimeout)
if err != nil {
logger.Error("Failed to connect to SFTP", "fileId", fileID.String(), "error", err)
return fmt.Errorf("failed to connect to SFTP: %w", err)
}
defer func() {
if closeErr := client.Close(); closeErr != nil {
logger.Error(
"Failed to close SFTP client",
"fileId",
fileID.String(),
"error",
closeErr,
)
}
if closeErr := sshConn.Close(); closeErr != nil {
logger.Error(
"Failed to close SSH connection",
"fileId",
fileID.String(),
"error",
closeErr,
)
}
}()
if s.Path != "" {
if err := s.ensureDirectory(client, s.Path); err != nil {
logger.Error(
"Failed to ensure directory",
"fileId",
fileID.String(),
"path",
s.Path,
"error",
err,
)
return fmt.Errorf("failed to ensure directory: %w", err)
}
}
filePath := s.getFilePath(fileID.String())
logger.Debug("Uploading file to SFTP", "fileId", fileID.String(), "filePath", filePath)
remoteFile, err := client.Create(filePath)
if err != nil {
logger.Error("Failed to create remote file", "fileId", fileID.String(), "error", err)
return fmt.Errorf("failed to create remote file: %w", err)
}
defer func() {
_ = remoteFile.Close()
}()
ctxReader := &contextReader{ctx: ctx, reader: file}
_, err = io.Copy(remoteFile, ctxReader)
if err != nil {
select {
case <-ctx.Done():
logger.Info("SFTP upload cancelled", "fileId", fileID.String())
return ctx.Err()
default:
logger.Error("Failed to upload file to SFTP", "fileId", fileID.String(), "error", err)
return fmt.Errorf("failed to upload file to SFTP: %w", err)
}
}
logger.Info(
"Successfully saved file to SFTP storage",
"fileId",
fileID.String(),
"filePath",
filePath,
)
return nil
}
func (s *SFTPStorage) GetFile(
encryptor encryption.FieldEncryptor,
fileID uuid.UUID,
) (io.ReadCloser, error) {
client, sshConn, err := s.connect(encryptor, sftpConnectTimeout)
if err != nil {
return nil, fmt.Errorf("failed to connect to SFTP: %w", err)
}
filePath := s.getFilePath(fileID.String())
remoteFile, err := client.Open(filePath)
if err != nil {
_ = client.Close()
_ = sshConn.Close()
return nil, fmt.Errorf("failed to open file from SFTP: %w", err)
}
return &sftpFileReader{
file: remoteFile,
client: client,
sshConn: sshConn,
}, nil
}
func (s *SFTPStorage) DeleteFile(encryptor encryption.FieldEncryptor, fileID uuid.UUID) error {
client, sshConn, err := s.connect(encryptor, sftpConnectTimeout)
if err != nil {
return fmt.Errorf("failed to connect to SFTP: %w", err)
}
defer func() {
_ = client.Close()
_ = sshConn.Close()
}()
filePath := s.getFilePath(fileID.String())
_, err = client.Stat(filePath)
if err != nil {
return nil
}
err = client.Remove(filePath)
if err != nil {
return fmt.Errorf("failed to delete file from SFTP: %w", err)
}
return nil
}
func (s *SFTPStorage) Validate(encryptor encryption.FieldEncryptor) error {
if s.Host == "" {
return errors.New("SFTP host is required")
}
if s.Username == "" {
return errors.New("SFTP username is required")
}
if s.Password == "" && s.PrivateKey == "" {
return errors.New("SFTP password or private key is required")
}
if s.Port <= 0 || s.Port > 65535 {
return errors.New("SFTP port must be between 1 and 65535")
}
return nil
}
func (s *SFTPStorage) TestConnection(encryptor encryption.FieldEncryptor) error {
ctx, cancel := context.WithTimeout(context.Background(), sftpTestConnectTimeout)
defer cancel()
client, sshConn, err := s.connectWithContext(ctx, encryptor, sftpTestConnectTimeout)
if err != nil {
return fmt.Errorf("failed to connect to SFTP: %w", err)
}
defer func() {
_ = client.Close()
_ = sshConn.Close()
}()
if s.Path != "" {
if err := s.ensureDirectory(client, s.Path); err != nil {
return fmt.Errorf("failed to access or create path '%s': %w", s.Path, err)
}
}
return nil
}
func (s *SFTPStorage) HideSensitiveData() {
s.Password = ""
s.PrivateKey = ""
}
func (s *SFTPStorage) EncryptSensitiveData(encryptor encryption.FieldEncryptor) error {
if s.Password != "" {
encrypted, err := encryptor.Encrypt(s.StorageID, s.Password)
if err != nil {
return fmt.Errorf("failed to encrypt SFTP password: %w", err)
}
s.Password = encrypted
}
if s.PrivateKey != "" {
encrypted, err := encryptor.Encrypt(s.StorageID, s.PrivateKey)
if err != nil {
return fmt.Errorf("failed to encrypt SFTP private key: %w", err)
}
s.PrivateKey = encrypted
}
return nil
}
func (s *SFTPStorage) Update(incoming *SFTPStorage) {
s.Host = incoming.Host
s.Port = incoming.Port
s.Username = incoming.Username
s.SkipHostKeyVerify = incoming.SkipHostKeyVerify
s.Path = incoming.Path
if incoming.Password != "" {
s.Password = incoming.Password
}
if incoming.PrivateKey != "" {
s.PrivateKey = incoming.PrivateKey
}
}
func (s *SFTPStorage) connect(
encryptor encryption.FieldEncryptor,
timeout time.Duration,
) (*sftp.Client, *ssh.Client, error) {
return s.connectWithContext(context.Background(), encryptor, timeout)
}
func (s *SFTPStorage) connectWithContext(
ctx context.Context,
encryptor encryption.FieldEncryptor,
timeout time.Duration,
) (*sftp.Client, *ssh.Client, error) {
var authMethods []ssh.AuthMethod
if s.Password != "" {
password, err := encryptor.Decrypt(s.StorageID, s.Password)
if err != nil {
return nil, nil, fmt.Errorf("failed to decrypt SFTP password: %w", err)
}
authMethods = append(authMethods, ssh.Password(password))
}
if s.PrivateKey != "" {
privateKey, err := encryptor.Decrypt(s.StorageID, s.PrivateKey)
if err != nil {
return nil, nil, fmt.Errorf("failed to decrypt SFTP private key: %w", err)
}
signer, err := ssh.ParsePrivateKey([]byte(privateKey))
if err != nil {
return nil, nil, fmt.Errorf("failed to parse private key: %w", err)
}
authMethods = append(authMethods, ssh.PublicKeys(signer))
}
var hostKeyCallback ssh.HostKeyCallback
if s.SkipHostKeyVerify {
hostKeyCallback = ssh.InsecureIgnoreHostKey()
} else {
hostKeyCallback = ssh.InsecureIgnoreHostKey()
}
config := &ssh.ClientConfig{
User: s.Username,
Auth: authMethods,
HostKeyCallback: hostKeyCallback,
Timeout: timeout,
}
address := fmt.Sprintf("%s:%d", s.Host, s.Port)
dialer := net.Dialer{Timeout: timeout}
conn, err := dialer.DialContext(ctx, "tcp", address)
if err != nil {
return nil, nil, fmt.Errorf("failed to dial SFTP server: %w", err)
}
sshConn, chans, reqs, err := ssh.NewClientConn(conn, address, config)
if err != nil {
_ = conn.Close()
return nil, nil, fmt.Errorf("failed to create SSH connection: %w", err)
}
sshClient := ssh.NewClient(sshConn, chans, reqs)
sftpClient, err := sftp.NewClient(sshClient)
if err != nil {
_ = sshClient.Close()
return nil, nil, fmt.Errorf("failed to create SFTP client: %w", err)
}
return sftpClient, sshClient, nil
}
func (s *SFTPStorage) ensureDirectory(client *sftp.Client, path string) error {
path = strings.TrimPrefix(path, "/")
path = strings.TrimSuffix(path, "/")
if path == "" {
return nil
}
parts := strings.Split(path, "/")
currentPath := ""
for _, part := range parts {
if part == "" || part == "." {
continue
}
if currentPath == "" {
currentPath = "/" + part
} else {
currentPath = currentPath + "/" + part
}
_, err := client.Stat(currentPath)
if err != nil {
err = client.Mkdir(currentPath)
if err != nil {
return fmt.Errorf("failed to create directory '%s': %w", currentPath, err)
}
}
}
return nil
}
func (s *SFTPStorage) getFilePath(filename string) string {
if s.Path == "" {
return filename
}
path := strings.TrimPrefix(s.Path, "/")
path = strings.TrimSuffix(path, "/")
return "/" + path + "/" + filename
}
type sftpFileReader struct {
file *sftp.File
client *sftp.Client
sshConn *ssh.Client
}
func (r *sftpFileReader) Read(p []byte) (n int, err error) {
return r.file.Read(p)
}
func (r *sftpFileReader) Close() error {
var errs []error
if r.file != nil {
if err := r.file.Close(); err != nil {
errs = append(errs, fmt.Errorf("failed to close file: %w", err))
}
}
if r.client != nil {
if err := r.client.Close(); err != nil {
errs = append(errs, fmt.Errorf("failed to close SFTP client: %w", err))
}
}
if r.sshConn != nil {
if err := r.sshConn.Close(); err != nil {
errs = append(errs, fmt.Errorf("failed to close SSH connection: %w", err))
}
}
if len(errs) > 0 {
return errs[0]
}
return nil
}
type contextReader struct {
ctx context.Context
reader io.Reader
}
func (r *contextReader) Read(p []byte) (n int, err error) {
select {
case <-r.ctx.Done():
return 0, r.ctx.Err()
default:
return r.reader.Read(p)
}
}

View File

@@ -34,17 +34,29 @@ func (r *StorageRepository) Save(storage *Storage) (*Storage, error) {
if storage.AzureBlobStorage != nil {
storage.AzureBlobStorage.StorageID = storage.ID
}
case StorageTypeFTP:
if storage.FTPStorage != nil {
storage.FTPStorage.StorageID = storage.ID
}
case StorageTypeSFTP:
if storage.SFTPStorage != nil {
storage.SFTPStorage.StorageID = storage.ID
}
case StorageTypeRclone:
if storage.RcloneStorage != nil {
storage.RcloneStorage.StorageID = storage.ID
}
}
if storage.ID == uuid.Nil {
if err := tx.Create(storage).
Omit("LocalStorage", "S3Storage", "GoogleDriveStorage", "NASStorage", "AzureBlobStorage").
Omit("LocalStorage", "S3Storage", "GoogleDriveStorage", "NASStorage", "AzureBlobStorage", "FTPStorage", "SFTPStorage", "RcloneStorage").
Error; err != nil {
return err
}
} else {
if err := tx.Save(storage).
Omit("LocalStorage", "S3Storage", "GoogleDriveStorage", "NASStorage", "AzureBlobStorage").
Omit("LocalStorage", "S3Storage", "GoogleDriveStorage", "NASStorage", "AzureBlobStorage", "FTPStorage", "SFTPStorage", "RcloneStorage").
Error; err != nil {
return err
}
@@ -86,6 +98,27 @@ func (r *StorageRepository) Save(storage *Storage) (*Storage, error) {
return err
}
}
case StorageTypeFTP:
if storage.FTPStorage != nil {
storage.FTPStorage.StorageID = storage.ID // Ensure ID is set
if err := tx.Save(storage.FTPStorage).Error; err != nil {
return err
}
}
case StorageTypeSFTP:
if storage.SFTPStorage != nil {
storage.SFTPStorage.StorageID = storage.ID // Ensure ID is set
if err := tx.Save(storage.SFTPStorage).Error; err != nil {
return err
}
}
case StorageTypeRclone:
if storage.RcloneStorage != nil {
storage.RcloneStorage.StorageID = storage.ID // Ensure ID is set
if err := tx.Save(storage.RcloneStorage).Error; err != nil {
return err
}
}
}
return nil
@@ -108,6 +141,9 @@ func (r *StorageRepository) FindByID(id uuid.UUID) (*Storage, error) {
Preload("GoogleDriveStorage").
Preload("NASStorage").
Preload("AzureBlobStorage").
Preload("FTPStorage").
Preload("SFTPStorage").
Preload("RcloneStorage").
Where("id = ?", id).
First(&s).Error; err != nil {
return nil, err
@@ -126,6 +162,9 @@ func (r *StorageRepository) FindByWorkspaceID(workspaceID uuid.UUID) ([]*Storage
Preload("GoogleDriveStorage").
Preload("NASStorage").
Preload("AzureBlobStorage").
Preload("FTPStorage").
Preload("SFTPStorage").
Preload("RcloneStorage").
Where("workspace_id = ?", workspaceID).
Order("name ASC").
Find(&storages).Error; err != nil {
@@ -169,6 +208,24 @@ func (r *StorageRepository) Delete(s *Storage) error {
return err
}
}
case StorageTypeFTP:
if s.FTPStorage != nil {
if err := tx.Delete(s.FTPStorage).Error; err != nil {
return err
}
}
case StorageTypeSFTP:
if s.SFTPStorage != nil {
if err := tx.Delete(s.SFTPStorage).Error; err != nil {
return err
}
}
case StorageTypeRclone:
if s.RcloneStorage != nil {
if err := tx.Delete(s.RcloneStorage).Error; err != nil {
return err
}
}
}
// Delete the main storage

View File

@@ -30,7 +30,6 @@ import (
workspaces_controllers "postgresus-backend/internal/features/workspaces/controllers"
workspaces_testing "postgresus-backend/internal/features/workspaces/testing"
test_utils "postgresus-backend/internal/util/testing"
"postgresus-backend/internal/util/tools"
)
const createAndFillTableQuery = `
@@ -114,6 +113,230 @@ func Test_BackupAndRestorePostgresqlWithEncryption_RestoreIsSuccessful(t *testin
}
}
func Test_BackupAndRestoreSupabase_PublicSchemaOnly_RestoreIsSuccessful(t *testing.T) {
env := config.GetEnv()
if env.TestSupabaseHost == "" {
t.Skip("Skipping Supabase test: missing environment variables")
}
portInt, err := strconv.Atoi(env.TestSupabasePort)
assert.NoError(t, err)
dsn := fmt.Sprintf(
"host=%s port=%d user=%s password=%s dbname=%s sslmode=require",
env.TestSupabaseHost,
portInt,
env.TestSupabaseUsername,
env.TestSupabasePassword,
env.TestSupabaseDatabase,
)
supabaseDB, err := sqlx.Connect("postgres", dsn)
assert.NoError(t, err)
defer supabaseDB.Close()
tableName := fmt.Sprintf("backup_test_%s", uuid.New().String()[:8])
createTableQuery := fmt.Sprintf(`
DROP TABLE IF EXISTS public.%s;
CREATE TABLE public.%s (
id SERIAL PRIMARY KEY,
name TEXT NOT NULL,
value INTEGER NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
INSERT INTO public.%s (name, value) VALUES
('test1', 100),
('test2', 200),
('test3', 300);
`, tableName, tableName, tableName)
_, err = supabaseDB.Exec(createTableQuery)
assert.NoError(t, err)
defer func() {
_, _ = supabaseDB.Exec(fmt.Sprintf(`DROP TABLE IF EXISTS public.%s`, tableName))
}()
router := createTestRouter()
user := users_testing.CreateTestUser(users_enums.UserRoleMember)
workspace := workspaces_testing.CreateTestWorkspace("Supabase Test Workspace", user, router)
storage := storages.CreateTestStorage(workspace.ID)
database := createSupabaseDatabaseViaAPI(
t, router, "Supabase Test Database", workspace.ID,
env.TestSupabaseHost, portInt,
env.TestSupabaseUsername, env.TestSupabasePassword, env.TestSupabaseDatabase,
[]string{"public"},
user.Token,
)
enableBackupsViaAPI(
t, router, database.ID, storage.ID,
backups_config.BackupEncryptionNone, user.Token,
)
createBackupViaAPI(t, router, database.ID, user.Token)
backup := waitForBackupCompletion(t, router, database.ID, user.Token, 5*time.Minute)
assert.Equal(t, backups.BackupStatusCompleted, backup.Status)
_, err = supabaseDB.Exec(fmt.Sprintf(`DELETE FROM public.%s`, tableName))
assert.NoError(t, err)
var countAfterDelete int
err = supabaseDB.Get(
&countAfterDelete,
fmt.Sprintf(`SELECT COUNT(*) FROM public.%s`, tableName),
)
assert.NoError(t, err)
assert.Equal(t, 0, countAfterDelete, "Table should be empty after delete")
createSupabaseRestoreViaAPI(
t, router, backup.ID,
env.TestSupabaseHost, portInt,
env.TestSupabaseUsername, env.TestSupabasePassword, env.TestSupabaseDatabase,
user.Token,
)
restore := waitForRestoreCompletion(t, router, backup.ID, user.Token, 5*time.Minute)
assert.Equal(t, restores_enums.RestoreStatusCompleted, restore.Status)
var countAfterRestore int
err = supabaseDB.Get(
&countAfterRestore,
fmt.Sprintf(`SELECT COUNT(*) FROM public.%s`, tableName),
)
assert.NoError(t, err)
assert.Equal(t, 3, countAfterRestore, "Table should have 3 rows after restore")
var restoredData []TestDataItem
err = supabaseDB.Select(
&restoredData,
fmt.Sprintf(`SELECT id, name, value, created_at FROM public.%s ORDER BY id`, tableName),
)
assert.NoError(t, err)
assert.Len(t, restoredData, 3)
assert.Equal(t, "test1", restoredData[0].Name)
assert.Equal(t, 100, restoredData[0].Value)
assert.Equal(t, "test2", restoredData[1].Name)
assert.Equal(t, 200, restoredData[1].Value)
assert.Equal(t, "test3", restoredData[2].Name)
assert.Equal(t, 300, restoredData[2].Value)
err = os.Remove(filepath.Join(config.GetEnv().DataFolder, backup.ID.String()))
if err != nil {
t.Logf("Warning: Failed to delete backup file: %v", err)
}
test_utils.MakeDeleteRequest(
t,
router,
"/api/v1/databases/"+database.ID.String(),
"Bearer "+user.Token,
http.StatusNoContent,
)
storages.RemoveTestStorage(storage.ID)
workspaces_testing.RemoveTestWorkspace(workspace, router)
}
func Test_BackupPostgresql_SchemaSelection_AllSchemasWhenNoneSpecified(t *testing.T) {
env := config.GetEnv()
cases := []struct {
name string
version string
port string
}{
{"PostgreSQL 12", "12", env.TestPostgres12Port},
{"PostgreSQL 13", "13", env.TestPostgres13Port},
{"PostgreSQL 14", "14", env.TestPostgres14Port},
{"PostgreSQL 15", "15", env.TestPostgres15Port},
{"PostgreSQL 16", "16", env.TestPostgres16Port},
{"PostgreSQL 17", "17", env.TestPostgres17Port},
{"PostgreSQL 18", "18", env.TestPostgres18Port},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
testSchemaSelectionAllSchemasForVersion(t, tc.version, tc.port)
})
}
}
func Test_BackupAndRestorePostgresql_WithExcludeExtensions_RestoreIsSuccessful(t *testing.T) {
env := config.GetEnv()
cases := []struct {
name string
version string
port string
}{
{"PostgreSQL 12", "12", env.TestPostgres12Port},
{"PostgreSQL 13", "13", env.TestPostgres13Port},
{"PostgreSQL 14", "14", env.TestPostgres14Port},
{"PostgreSQL 15", "15", env.TestPostgres15Port},
{"PostgreSQL 16", "16", env.TestPostgres16Port},
{"PostgreSQL 17", "17", env.TestPostgres17Port},
{"PostgreSQL 18", "18", env.TestPostgres18Port},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
testBackupRestoreWithExcludeExtensionsForVersion(t, tc.version, tc.port)
})
}
}
func Test_BackupAndRestorePostgresql_WithoutExcludeExtensions_ExtensionsAreRecovered(t *testing.T) {
env := config.GetEnv()
cases := []struct {
name string
version string
port string
}{
{"PostgreSQL 12", "12", env.TestPostgres12Port},
{"PostgreSQL 13", "13", env.TestPostgres13Port},
{"PostgreSQL 14", "14", env.TestPostgres14Port},
{"PostgreSQL 15", "15", env.TestPostgres15Port},
{"PostgreSQL 16", "16", env.TestPostgres16Port},
{"PostgreSQL 17", "17", env.TestPostgres17Port},
{"PostgreSQL 18", "18", env.TestPostgres18Port},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
testBackupRestoreWithoutExcludeExtensionsForVersion(t, tc.version, tc.port)
})
}
}
func Test_BackupPostgresql_SchemaSelection_OnlySpecifiedSchemas(t *testing.T) {
env := config.GetEnv()
cases := []struct {
name string
version string
port string
}{
{"PostgreSQL 12", "12", env.TestPostgres12Port},
{"PostgreSQL 13", "13", env.TestPostgres13Port},
{"PostgreSQL 14", "14", env.TestPostgres14Port},
{"PostgreSQL 15", "15", env.TestPostgres15Port},
{"PostgreSQL 16", "16", env.TestPostgres16Port},
{"PostgreSQL 17", "17", env.TestPostgres17Port},
{"PostgreSQL 18", "18", env.TestPostgres18Port},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
testSchemaSelectionOnlySpecifiedSchemasForVersion(t, tc.version, tc.port)
})
}
}
func testBackupRestoreForVersion(t *testing.T, pgVersion string, port string) {
container, err := connectToPostgresContainer(pgVersion, port)
assert.NoError(t, err)
@@ -132,10 +355,9 @@ func testBackupRestoreForVersion(t *testing.T, pgVersion string, port string) {
storage := storages.CreateTestStorage(workspace.ID)
pgVersionEnum := tools.GetPostgresqlVersionEnum(pgVersion)
database := createDatabaseViaAPI(
t, router, "Test Database", workspace.ID,
pgVersionEnum, container.Host, container.Port,
container.Host, container.Port,
container.Username, container.Password, container.Database,
user.Token,
)
@@ -164,7 +386,7 @@ func testBackupRestoreForVersion(t *testing.T, pgVersion string, port string) {
defer newDB.Close()
createRestoreViaAPI(
t, router, backup.ID, pgVersionEnum,
t, router, backup.ID,
container.Host, container.Port,
container.Username, container.Password, newDBName,
user.Token,
@@ -199,6 +421,522 @@ func testBackupRestoreForVersion(t *testing.T, pgVersion string, port string) {
workspaces_testing.RemoveTestWorkspace(workspace, router)
}
func testSchemaSelectionAllSchemasForVersion(t *testing.T, pgVersion string, port string) {
container, err := connectToPostgresContainer(pgVersion, port)
assert.NoError(t, err)
defer container.DB.Close()
_, err = container.DB.Exec(`
DROP SCHEMA IF EXISTS schema_a CASCADE;
DROP SCHEMA IF EXISTS schema_b CASCADE;
CREATE SCHEMA schema_a;
CREATE SCHEMA schema_b;
CREATE TABLE public.public_table (id SERIAL PRIMARY KEY, data TEXT);
CREATE TABLE schema_a.table_a (id SERIAL PRIMARY KEY, data TEXT);
CREATE TABLE schema_b.table_b (id SERIAL PRIMARY KEY, data TEXT);
INSERT INTO public.public_table (data) VALUES ('public_data');
INSERT INTO schema_a.table_a (data) VALUES ('schema_a_data');
INSERT INTO schema_b.table_b (data) VALUES ('schema_b_data');
`)
assert.NoError(t, err)
defer func() {
_, _ = container.DB.Exec(`
DROP TABLE IF EXISTS public.public_table;
DROP SCHEMA IF EXISTS schema_a CASCADE;
DROP SCHEMA IF EXISTS schema_b CASCADE;
`)
}()
router := createTestRouter()
user := users_testing.CreateTestUser(users_enums.UserRoleMember)
workspace := workspaces_testing.CreateTestWorkspace("Schema Test Workspace", user, router)
storage := storages.CreateTestStorage(workspace.ID)
database := createDatabaseWithSchemasViaAPI(
t, router, "All Schemas Database", workspace.ID,
container.Host, container.Port,
container.Username, container.Password, container.Database,
nil,
user.Token,
)
enableBackupsViaAPI(
t, router, database.ID, storage.ID,
backups_config.BackupEncryptionNone, user.Token,
)
createBackupViaAPI(t, router, database.ID, user.Token)
backup := waitForBackupCompletion(t, router, database.ID, user.Token, 5*time.Minute)
assert.Equal(t, backups.BackupStatusCompleted, backup.Status)
newDBName := "restored_all_schemas_" + pgVersion
_, err = container.DB.Exec(fmt.Sprintf("DROP DATABASE IF EXISTS %s;", newDBName))
assert.NoError(t, err)
_, err = container.DB.Exec(fmt.Sprintf("CREATE DATABASE %s;", newDBName))
assert.NoError(t, err)
newDSN := fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=disable",
container.Host, container.Port, container.Username, container.Password, newDBName)
newDB, err := sqlx.Connect("postgres", newDSN)
assert.NoError(t, err)
defer newDB.Close()
createRestoreViaAPI(
t, router, backup.ID,
container.Host, container.Port,
container.Username, container.Password, newDBName,
user.Token,
)
restore := waitForRestoreCompletion(t, router, backup.ID, user.Token, 5*time.Minute)
assert.Equal(t, restores_enums.RestoreStatusCompleted, restore.Status)
var publicTableExists bool
err = newDB.Get(&publicTableExists, `
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = 'public_table'
)
`)
assert.NoError(t, err)
assert.True(t, publicTableExists, "public.public_table should exist in restored database")
var schemaATableExists bool
err = newDB.Get(&schemaATableExists, `
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'schema_a' AND table_name = 'table_a'
)
`)
assert.NoError(t, err)
assert.True(t, schemaATableExists, "schema_a.table_a should exist in restored database")
var schemaBTableExists bool
err = newDB.Get(&schemaBTableExists, `
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'schema_b' AND table_name = 'table_b'
)
`)
assert.NoError(t, err)
assert.True(t, schemaBTableExists, "schema_b.table_b should exist in restored database")
err = os.Remove(filepath.Join(config.GetEnv().DataFolder, backup.ID.String()))
if err != nil {
t.Logf("Warning: Failed to delete backup file: %v", err)
}
test_utils.MakeDeleteRequest(
t,
router,
"/api/v1/databases/"+database.ID.String(),
"Bearer "+user.Token,
http.StatusNoContent,
)
storages.RemoveTestStorage(storage.ID)
workspaces_testing.RemoveTestWorkspace(workspace, router)
}
func testBackupRestoreWithExcludeExtensionsForVersion(t *testing.T, pgVersion string, port string) {
container, err := connectToPostgresContainer(pgVersion, port)
assert.NoError(t, err)
defer container.DB.Close()
// Create table with uuid-ossp extension and add a comment on the extension
// The comment is important to test that COMMENT ON EXTENSION statements are also excluded
_, err = container.DB.Exec(`
DROP EXTENSION IF EXISTS "uuid-ossp" CASCADE;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
COMMENT ON EXTENSION "uuid-ossp" IS 'Test comment on uuid-ossp extension';
DROP TABLE IF EXISTS test_extension_data;
CREATE TABLE test_extension_data (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
name TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
INSERT INTO test_extension_data (name) VALUES ('test1'), ('test2'), ('test3');
`)
assert.NoError(t, err)
defer func() {
_, _ = container.DB.Exec(`
DROP TABLE IF EXISTS test_extension_data;
DROP EXTENSION IF EXISTS "uuid-ossp" CASCADE;
`)
}()
router := createTestRouter()
user := users_testing.CreateTestUser(users_enums.UserRoleMember)
workspace := workspaces_testing.CreateTestWorkspace("Extension Test Workspace", user, router)
storage := storages.CreateTestStorage(workspace.ID)
database := createDatabaseViaAPI(
t, router, "Extension Test Database", workspace.ID,
container.Host, container.Port,
container.Username, container.Password, container.Database,
user.Token,
)
enableBackupsViaAPI(
t, router, database.ID, storage.ID,
backups_config.BackupEncryptionNone, user.Token,
)
createBackupViaAPI(t, router, database.ID, user.Token)
backup := waitForBackupCompletion(t, router, database.ID, user.Token, 5*time.Minute)
assert.Equal(t, backups.BackupStatusCompleted, backup.Status)
// Create new database for restore with extension pre-installed
newDBName := "restored_exclude_ext_" + pgVersion
_, err = container.DB.Exec(fmt.Sprintf("DROP DATABASE IF EXISTS %s;", newDBName))
assert.NoError(t, err)
_, err = container.DB.Exec(fmt.Sprintf("CREATE DATABASE %s;", newDBName))
assert.NoError(t, err)
newDSN := fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=disable",
container.Host, container.Port, container.Username, container.Password, newDBName)
newDB, err := sqlx.Connect("postgres", newDSN)
assert.NoError(t, err)
defer newDB.Close()
// Pre-install the extension in the target database (simulating managed service behavior)
_, err = newDB.Exec(`CREATE EXTENSION IF NOT EXISTS "uuid-ossp";`)
assert.NoError(t, err)
// Restore with isExcludeExtensions=true
createRestoreWithOptionsViaAPI(
t, router, backup.ID,
container.Host, container.Port,
container.Username, container.Password, newDBName,
true, // isExcludeExtensions
user.Token,
)
restore := waitForRestoreCompletion(t, router, backup.ID, user.Token, 5*time.Minute)
assert.Equal(t, restores_enums.RestoreStatusCompleted, restore.Status)
// Verify the table was restored
var tableExists bool
err = newDB.Get(&tableExists, `
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = 'test_extension_data'
)
`)
assert.NoError(t, err)
assert.True(t, tableExists, "test_extension_data should exist in restored database")
// Verify data was restored
var count int
err = newDB.Get(&count, `SELECT COUNT(*) FROM test_extension_data`)
assert.NoError(t, err)
assert.Equal(t, 3, count, "Should have 3 rows after restore")
// Verify extension still works (uuid_generate_v4 should work)
var newUUID string
err = newDB.Get(&newUUID, `SELECT uuid_generate_v4()::text`)
assert.NoError(t, err)
assert.NotEmpty(t, newUUID, "uuid_generate_v4 should work")
// Cleanup
err = os.Remove(filepath.Join(config.GetEnv().DataFolder, backup.ID.String()))
if err != nil {
t.Logf("Warning: Failed to delete backup file: %v", err)
}
test_utils.MakeDeleteRequest(
t,
router,
"/api/v1/databases/"+database.ID.String(),
"Bearer "+user.Token,
http.StatusNoContent,
)
storages.RemoveTestStorage(storage.ID)
workspaces_testing.RemoveTestWorkspace(workspace, router)
}
func testBackupRestoreWithoutExcludeExtensionsForVersion(
t *testing.T,
pgVersion string,
port string,
) {
container, err := connectToPostgresContainer(pgVersion, port)
assert.NoError(t, err)
defer container.DB.Close()
// Create table with uuid-ossp extension
_, err = container.DB.Exec(`
DROP EXTENSION IF EXISTS "uuid-ossp" CASCADE;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
DROP TABLE IF EXISTS test_extension_recovery;
CREATE TABLE test_extension_recovery (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
name TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
INSERT INTO test_extension_recovery (name) VALUES ('test1'), ('test2'), ('test3');
`)
assert.NoError(t, err)
defer func() {
_, _ = container.DB.Exec(`
DROP TABLE IF EXISTS test_extension_recovery;
DROP EXTENSION IF EXISTS "uuid-ossp" CASCADE;
`)
}()
router := createTestRouter()
user := users_testing.CreateTestUser(users_enums.UserRoleMember)
workspace := workspaces_testing.CreateTestWorkspace(
"Extension Recovery Test Workspace",
user,
router,
)
storage := storages.CreateTestStorage(workspace.ID)
database := createDatabaseViaAPI(
t, router, "Extension Recovery Test Database", workspace.ID,
container.Host, container.Port,
container.Username, container.Password, container.Database,
user.Token,
)
enableBackupsViaAPI(
t, router, database.ID, storage.ID,
backups_config.BackupEncryptionNone, user.Token,
)
createBackupViaAPI(t, router, database.ID, user.Token)
backup := waitForBackupCompletion(t, router, database.ID, user.Token, 5*time.Minute)
assert.Equal(t, backups.BackupStatusCompleted, backup.Status)
// Create new database for restore WITHOUT pre-installed extension
newDBName := "restored_with_ext_" + pgVersion
_, err = container.DB.Exec(fmt.Sprintf("DROP DATABASE IF EXISTS %s;", newDBName))
assert.NoError(t, err)
_, err = container.DB.Exec(fmt.Sprintf("CREATE DATABASE %s;", newDBName))
assert.NoError(t, err)
newDSN := fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=disable",
container.Host, container.Port, container.Username, container.Password, newDBName)
newDB, err := sqlx.Connect("postgres", newDSN)
assert.NoError(t, err)
defer newDB.Close()
// Verify extension does NOT exist before restore
var extensionExistsBefore bool
err = newDB.Get(&extensionExistsBefore, `
SELECT EXISTS (
SELECT FROM pg_extension WHERE extname = 'uuid-ossp'
)
`)
assert.NoError(t, err)
assert.False(t, extensionExistsBefore, "Extension should NOT exist before restore")
// Restore with isExcludeExtensions=false (extensions should be recovered)
createRestoreWithOptionsViaAPI(
t, router, backup.ID,
container.Host, container.Port,
container.Username, container.Password, newDBName,
false, // isExcludeExtensions = false means extensions ARE included
user.Token,
)
restore := waitForRestoreCompletion(t, router, backup.ID, user.Token, 5*time.Minute)
assert.Equal(t, restores_enums.RestoreStatusCompleted, restore.Status)
// Verify the extension was recovered
var extensionExists bool
err = newDB.Get(&extensionExists, `
SELECT EXISTS (
SELECT FROM pg_extension WHERE extname = 'uuid-ossp'
)
`)
assert.NoError(t, err)
assert.True(t, extensionExists, "Extension 'uuid-ossp' should be recovered during restore")
// Verify the table was restored
var tableExists bool
err = newDB.Get(&tableExists, `
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = 'test_extension_recovery'
)
`)
assert.NoError(t, err)
assert.True(t, tableExists, "test_extension_recovery should exist in restored database")
// Verify data was restored
var count int
err = newDB.Get(&count, `SELECT COUNT(*) FROM test_extension_recovery`)
assert.NoError(t, err)
assert.Equal(t, 3, count, "Should have 3 rows after restore")
// Verify extension works (uuid_generate_v4 should work)
var newUUID string
err = newDB.Get(&newUUID, `SELECT uuid_generate_v4()::text`)
assert.NoError(t, err)
assert.NotEmpty(t, newUUID, "uuid_generate_v4 should work after extension recovery")
// Cleanup
err = os.Remove(filepath.Join(config.GetEnv().DataFolder, backup.ID.String()))
if err != nil {
t.Logf("Warning: Failed to delete backup file: %v", err)
}
test_utils.MakeDeleteRequest(
t,
router,
"/api/v1/databases/"+database.ID.String(),
"Bearer "+user.Token,
http.StatusNoContent,
)
storages.RemoveTestStorage(storage.ID)
workspaces_testing.RemoveTestWorkspace(workspace, router)
}
func testSchemaSelectionOnlySpecifiedSchemasForVersion(
t *testing.T,
pgVersion string,
port string,
) {
container, err := connectToPostgresContainer(pgVersion, port)
assert.NoError(t, err)
defer container.DB.Close()
_, err = container.DB.Exec(`
DROP SCHEMA IF EXISTS schema_a CASCADE;
DROP SCHEMA IF EXISTS schema_b CASCADE;
CREATE SCHEMA schema_a;
CREATE SCHEMA schema_b;
CREATE TABLE public.public_table (id SERIAL PRIMARY KEY, data TEXT);
CREATE TABLE schema_a.table_a (id SERIAL PRIMARY KEY, data TEXT);
CREATE TABLE schema_b.table_b (id SERIAL PRIMARY KEY, data TEXT);
INSERT INTO public.public_table (data) VALUES ('public_data');
INSERT INTO schema_a.table_a (data) VALUES ('schema_a_data');
INSERT INTO schema_b.table_b (data) VALUES ('schema_b_data');
`)
assert.NoError(t, err)
defer func() {
_, _ = container.DB.Exec(`
DROP TABLE IF EXISTS public.public_table;
DROP SCHEMA IF EXISTS schema_a CASCADE;
DROP SCHEMA IF EXISTS schema_b CASCADE;
`)
}()
router := createTestRouter()
user := users_testing.CreateTestUser(users_enums.UserRoleMember)
workspace := workspaces_testing.CreateTestWorkspace("Schema Test Workspace", user, router)
storage := storages.CreateTestStorage(workspace.ID)
database := createDatabaseWithSchemasViaAPI(
t, router, "Specific Schemas Database", workspace.ID,
container.Host, container.Port,
container.Username, container.Password, container.Database,
[]string{"public", "schema_a"},
user.Token,
)
enableBackupsViaAPI(
t, router, database.ID, storage.ID,
backups_config.BackupEncryptionNone, user.Token,
)
createBackupViaAPI(t, router, database.ID, user.Token)
backup := waitForBackupCompletion(t, router, database.ID, user.Token, 5*time.Minute)
assert.Equal(t, backups.BackupStatusCompleted, backup.Status)
newDBName := "restored_specific_schemas_" + pgVersion
_, err = container.DB.Exec(fmt.Sprintf("DROP DATABASE IF EXISTS %s;", newDBName))
assert.NoError(t, err)
_, err = container.DB.Exec(fmt.Sprintf("CREATE DATABASE %s;", newDBName))
assert.NoError(t, err)
newDSN := fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=disable",
container.Host, container.Port, container.Username, container.Password, newDBName)
newDB, err := sqlx.Connect("postgres", newDSN)
assert.NoError(t, err)
defer newDB.Close()
createRestoreViaAPI(
t, router, backup.ID,
container.Host, container.Port,
container.Username, container.Password, newDBName,
user.Token,
)
restore := waitForRestoreCompletion(t, router, backup.ID, user.Token, 5*time.Minute)
assert.Equal(t, restores_enums.RestoreStatusCompleted, restore.Status)
var publicTableExists bool
err = newDB.Get(&publicTableExists, `
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = 'public_table'
)
`)
assert.NoError(t, err)
assert.True(t, publicTableExists, "public.public_table should exist (was included)")
var schemaATableExists bool
err = newDB.Get(&schemaATableExists, `
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'schema_a' AND table_name = 'table_a'
)
`)
assert.NoError(t, err)
assert.True(t, schemaATableExists, "schema_a.table_a should exist (was included)")
var schemaBTableExists bool
err = newDB.Get(&schemaBTableExists, `
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'schema_b' AND table_name = 'table_b'
)
`)
assert.NoError(t, err)
assert.False(t, schemaBTableExists, "schema_b.table_b should NOT exist (was excluded)")
err = os.Remove(filepath.Join(config.GetEnv().DataFolder, backup.ID.String()))
if err != nil {
t.Logf("Warning: Failed to delete backup file: %v", err)
}
test_utils.MakeDeleteRequest(
t,
router,
"/api/v1/databases/"+database.ID.String(),
"Bearer "+user.Token,
http.StatusNoContent,
)
storages.RemoveTestStorage(storage.ID)
workspaces_testing.RemoveTestWorkspace(workspace, router)
}
func testBackupRestoreWithEncryptionForVersion(t *testing.T, pgVersion string, port string) {
container, err := connectToPostgresContainer(pgVersion, port)
assert.NoError(t, err)
@@ -217,10 +955,9 @@ func testBackupRestoreWithEncryptionForVersion(t *testing.T, pgVersion string, p
storage := storages.CreateTestStorage(workspace.ID)
pgVersionEnum := tools.GetPostgresqlVersionEnum(pgVersion)
database := createDatabaseViaAPI(
t, router, "Test Database", workspace.ID,
pgVersionEnum, container.Host, container.Port,
container.Host, container.Port,
container.Username, container.Password, container.Database,
user.Token,
)
@@ -250,7 +987,7 @@ func testBackupRestoreWithEncryptionForVersion(t *testing.T, pgVersion string, p
defer newDB.Close()
createRestoreViaAPI(
t, router, backup.ID, pgVersionEnum,
t, router, backup.ID,
container.Host, container.Port,
container.Username, container.Password, newDBName,
user.Token,
@@ -379,7 +1116,6 @@ func createDatabaseViaAPI(
router *gin.Engine,
name string,
workspaceID uuid.UUID,
pgVersion tools.PostgresqlVersion,
host string,
port int,
username string,
@@ -392,7 +1128,6 @@ func createDatabaseViaAPI(
WorkspaceID: &workspaceID,
Type: databases.DatabaseTypePostgres,
Postgresql: &pgtypes.PostgresqlDatabase{
Version: pgVersion,
Host: host,
Port: port,
Username: username,
@@ -475,7 +1210,167 @@ func createRestoreViaAPI(
t *testing.T,
router *gin.Engine,
backupID uuid.UUID,
pgVersion tools.PostgresqlVersion,
host string,
port int,
username string,
password string,
database string,
token string,
) {
createRestoreWithOptionsViaAPI(
t,
router,
backupID,
host,
port,
username,
password,
database,
false,
token,
)
}
func createRestoreWithOptionsViaAPI(
t *testing.T,
router *gin.Engine,
backupID uuid.UUID,
host string,
port int,
username string,
password string,
database string,
isExcludeExtensions bool,
token string,
) {
request := restores.RestoreBackupRequest{
PostgresqlDatabase: &pgtypes.PostgresqlDatabase{
Host: host,
Port: port,
Username: username,
Password: password,
Database: &database,
IsExcludeExtensions: isExcludeExtensions,
},
}
test_utils.MakePostRequest(
t,
router,
fmt.Sprintf("/api/v1/restores/%s/restore", backupID.String()),
"Bearer "+token,
request,
http.StatusOK,
)
}
func createDatabaseWithSchemasViaAPI(
t *testing.T,
router *gin.Engine,
name string,
workspaceID uuid.UUID,
host string,
port int,
username string,
password string,
database string,
includeSchemas []string,
token string,
) *databases.Database {
request := databases.Database{
Name: name,
WorkspaceID: &workspaceID,
Type: databases.DatabaseTypePostgres,
Postgresql: &pgtypes.PostgresqlDatabase{
Host: host,
Port: port,
Username: username,
Password: password,
Database: &database,
IncludeSchemas: includeSchemas,
},
}
w := workspaces_testing.MakeAPIRequest(
router,
"POST",
"/api/v1/databases/create",
"Bearer "+token,
request,
)
if w.Code != http.StatusCreated {
t.Fatalf(
"Failed to create database with schemas. Status: %d, Body: %s",
w.Code,
w.Body.String(),
)
}
var createdDatabase databases.Database
if err := json.Unmarshal(w.Body.Bytes(), &createdDatabase); err != nil {
t.Fatalf("Failed to unmarshal database response: %v", err)
}
return &createdDatabase
}
func createSupabaseDatabaseViaAPI(
t *testing.T,
router *gin.Engine,
name string,
workspaceID uuid.UUID,
host string,
port int,
username string,
password string,
database string,
includeSchemas []string,
token string,
) *databases.Database {
request := databases.Database{
Name: name,
WorkspaceID: &workspaceID,
Type: databases.DatabaseTypePostgres,
Postgresql: &pgtypes.PostgresqlDatabase{
Host: host,
Port: port,
Username: username,
Password: password,
Database: &database,
IsHttps: true,
IncludeSchemas: includeSchemas,
},
}
w := workspaces_testing.MakeAPIRequest(
router,
"POST",
"/api/v1/databases/create",
"Bearer "+token,
request,
)
if w.Code != http.StatusCreated {
t.Fatalf(
"Failed to create Supabase database. Status: %d, Body: %s",
w.Code,
w.Body.String(),
)
}
var createdDatabase databases.Database
if err := json.Unmarshal(w.Body.Bytes(), &createdDatabase); err != nil {
t.Fatalf("Failed to unmarshal database response: %v", err)
}
return &createdDatabase
}
func createSupabaseRestoreViaAPI(
t *testing.T,
router *gin.Engine,
backupID uuid.UUID,
host string,
port int,
username string,
@@ -485,12 +1380,12 @@ func createRestoreViaAPI(
) {
request := restores.RestoreBackupRequest{
PostgresqlDatabase: &pgtypes.PostgresqlDatabase{
Version: pgVersion,
Host: host,
Port: port,
Username: username,
Password: password,
Database: &database,
IsHttps: true,
},
}
@@ -550,7 +1445,6 @@ func connectToPostgresContainer(version string, port string) (*PostgresContainer
Username: username,
Password: password,
Database: dbName,
Version: version,
DB: db,
}, nil
}

View File

@@ -6,6 +6,7 @@ import (
"os"
"path/filepath"
"runtime"
"strings"
env_utils "postgresus-backend/internal/util/env"
)
@@ -151,6 +152,24 @@ func VerifyPostgresesInstallation(
logger.Info("All PostgreSQL version-specific client tools verification completed successfully!")
}
// EscapePgpassField escapes special characters in a field value for .pgpass file format.
// According to PostgreSQL documentation, the .pgpass file format requires:
// - Backslash (\) must be escaped as \\
// - Colon (:) must be escaped as \:
// Additionally, newlines and carriage returns are removed to prevent format corruption.
func EscapePgpassField(field string) string {
// Remove newlines and carriage returns that would break .pgpass format
field = strings.ReplaceAll(field, "\r", "")
field = strings.ReplaceAll(field, "\n", "")
// Escape backslashes first (order matters!)
// Then escape colons
field = strings.ReplaceAll(field, "\\", "\\\\")
field = strings.ReplaceAll(field, ":", "\\:")
return field
}
func getPostgresqlBasePath(
version PostgresqlVersion,
envMode env_utils.EnvMode,

View File

@@ -0,0 +1,18 @@
-- +goose Up
-- +goose StatementBegin
ALTER TABLE webhook_notifiers
ADD COLUMN body_template TEXT,
ADD COLUMN headers TEXT DEFAULT '[]';
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
ALTER TABLE webhook_notifiers
DROP COLUMN body_template,
DROP COLUMN headers;
-- +goose StatementEnd

View File

@@ -0,0 +1,11 @@
-- +goose Up
-- +goose StatementBegin
ALTER TABLE postgresql_databases
ADD COLUMN include_schemas TEXT NOT NULL DEFAULT '';
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
ALTER TABLE postgresql_databases
DROP COLUMN include_schemas;
-- +goose StatementEnd

View File

@@ -0,0 +1,11 @@
-- +goose Up
-- +goose StatementBegin
ALTER TABLE s3_storages
ADD COLUMN skip_tls_verify BOOLEAN NOT NULL DEFAULT FALSE;
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
ALTER TABLE s3_storages
DROP COLUMN skip_tls_verify;
-- +goose StatementEnd

View File

@@ -0,0 +1,29 @@
-- +goose Up
-- +goose StatementBegin
CREATE TABLE ftp_storages (
storage_id UUID PRIMARY KEY,
host TEXT NOT NULL,
port INTEGER NOT NULL DEFAULT 21,
username TEXT NOT NULL,
password TEXT NOT NULL,
path TEXT,
use_ssl BOOLEAN NOT NULL DEFAULT FALSE,
skip_tls_verify BOOLEAN NOT NULL DEFAULT FALSE,
passive_mode BOOLEAN NOT NULL DEFAULT TRUE
);
ALTER TABLE ftp_storages
ADD CONSTRAINT fk_ftp_storages_storage
FOREIGN KEY (storage_id)
REFERENCES storages (id)
ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED;
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE IF EXISTS ftp_storages;
-- +goose StatementEnd

View File

@@ -0,0 +1,15 @@
-- +goose Up
-- +goose StatementBegin
ALTER TABLE ftp_storages
DROP COLUMN passive_mode;
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
ALTER TABLE ftp_storages
ADD COLUMN passive_mode BOOLEAN NOT NULL DEFAULT TRUE;
-- +goose StatementEnd

View File

@@ -0,0 +1,23 @@
-- +goose Up
-- +goose StatementBegin
CREATE TABLE rclone_storages (
storage_id UUID PRIMARY KEY,
config_content TEXT NOT NULL,
remote_path TEXT
);
ALTER TABLE rclone_storages
ADD CONSTRAINT fk_rclone_storages_storage
FOREIGN KEY (storage_id)
REFERENCES storages (id)
ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED;
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE IF EXISTS rclone_storages;
-- +goose StatementEnd

View File

@@ -0,0 +1,28 @@
-- +goose Up
-- +goose StatementBegin
CREATE TABLE sftp_storages (
storage_id UUID PRIMARY KEY,
host TEXT NOT NULL,
port INTEGER NOT NULL DEFAULT 22,
username TEXT NOT NULL,
password TEXT,
private_key TEXT,
path TEXT,
skip_host_key_verify BOOLEAN NOT NULL DEFAULT FALSE
);
ALTER TABLE sftp_storages
ADD CONSTRAINT fk_sftp_storages_storage
FOREIGN KEY (storage_id)
REFERENCES storages (id)
ON DELETE CASCADE DEFERRABLE INITIALLY DEFERRED;
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE IF EXISTS sftp_storages;
-- +goose StatementEnd

23
deploy/helm/.helmignore Normal file
View File

@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/

22
deploy/helm/Chart.yaml Normal file
View File

@@ -0,0 +1,22 @@
apiVersion: v2
name: postgresus
description: A Helm chart for Postgresus - PostgreSQL backup and management system
type: application
version: 0.0.0
appVersion: "latest"
keywords:
- postgresql
- backup
- database
- restore
home: https://github.com/RostislavDugin/postgresus
sources:
- https://github.com/RostislavDugin/postgresus
- https://github.com/RostislavDugin/postgresus/tree/main/deploy/helm
maintainers:
- name: Rostislav Dugin
url: https://github.com/RostislavDugin
icon: https://raw.githubusercontent.com/RostislavDugin/postgresus/main/frontend/public/logo.svg

245
deploy/helm/README.md Normal file
View File

@@ -0,0 +1,245 @@
# Postgresus Helm Chart
## Installation
Install directly from the OCI registry (no need to clone the repository):
```bash
helm install postgresus oci://ghcr.io/rostislavdugin/charts/postgresus \
-n postgresus --create-namespace
```
The `-n postgresus --create-namespace` flags control which namespace the chart is installed into. You can use any namespace name you prefer.
## Accessing Postgresus
By default, the chart creates a ClusterIP service. Use port-forward to access:
```bash
kubectl port-forward svc/postgresus-service 4005:4005 -n postgresus
```
Then open `http://localhost:4005` in your browser.
## Configuration
### Main Parameters
| Parameter | Description | Default Value |
| ------------------ | ------------------ | --------------------------- |
| `image.repository` | Docker image | `rostislavdugin/postgresus` |
| `image.tag` | Image tag | `latest` |
| `image.pullPolicy` | Image pull policy | `Always` |
| `replicaCount` | Number of replicas | `1` |
### Custom Root CA
| Parameter | Description | Default Value |
| -------------- | ---------------------------------------- | ------------- |
| `customRootCA` | Name of Secret containing CA certificate | `""` |
To trust a custom CA certificate (e.g., for internal services with self-signed certificates):
1. Create a Secret with your CA certificate:
```bash
kubectl create secret generic my-root-ca \
--from-file=ca.crt=./path/to/ca-certificate.crt
```
2. Reference it in values:
```yaml
customRootCA: my-root-ca
```
The certificate will be mounted to `/etc/ssl/certs/custom-root-ca.crt` and the `SSL_CERT_FILE` environment variable will be set automatically.
### Service
| Parameter | Description | Default Value |
| -------------------------- | ----------------------- | ------------- |
| `service.type` | Service type | `ClusterIP` |
| `service.port` | Service port | `4005` |
| `service.targetPort` | Container port | `4005` |
| `service.headless.enabled` | Enable headless service | `true` |
### Storage
| Parameter | Description | Default Value |
| ------------------------------ | ------------------------- | ---------------------- |
| `persistence.enabled` | Enable persistent storage | `true` |
| `persistence.storageClassName` | Storage class | `""` (cluster default) |
| `persistence.accessMode` | Access mode | `ReadWriteOnce` |
| `persistence.size` | Storage size | `10Gi` |
| `persistence.mountPath` | Mount path | `/postgresus-data` |
### Resources
| Parameter | Description | Default Value |
| --------------------------- | -------------- | ------------- |
| `resources.requests.memory` | Memory request | `1Gi` |
| `resources.requests.cpu` | CPU request | `500m` |
| `resources.limits.memory` | Memory limit | `1Gi` |
| `resources.limits.cpu` | CPU limit | `500m` |
## External Access Options
### Option 1: Port Forward (Default)
Best for development or quick access:
```bash
kubectl port-forward svc/postgresus-service 4005:4005 -n postgresus
```
Access at `http://localhost:4005`
### Option 2: NodePort
For direct access via node IP:
```yaml
# nodeport-values.yaml
service:
type: NodePort
port: 4005
targetPort: 4005
nodePort: 30080
```
```bash
helm install postgresus oci://ghcr.io/rostislavdugin/charts/postgresus \
-n postgresus --create-namespace \
-f nodeport-values.yaml
```
Access at `http://<NODE-IP>:30080`
### Option 3: LoadBalancer
For cloud environments with load balancer support:
```yaml
# loadbalancer-values.yaml
service:
type: LoadBalancer
port: 80
targetPort: 4005
```
```bash
helm install postgresus oci://ghcr.io/rostislavdugin/charts/postgresus \
-n postgresus --create-namespace \
-f loadbalancer-values.yaml
```
Get the external IP:
```bash
kubectl get svc -n postgresus
```
Access at `http://<EXTERNAL-IP>`
### Option 4: Ingress
For domain-based access with TLS:
```yaml
# ingress-values.yaml
ingress:
enabled: true
className: nginx
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
hosts:
- host: backup.example.com
paths:
- path: /
pathType: Prefix
tls:
- secretName: backup-example-com-tls
hosts:
- backup.example.com
```
```bash
helm install postgresus oci://ghcr.io/rostislavdugin/charts/postgresus \
-n postgresus --create-namespace \
-f ingress-values.yaml
```
### Option 5: HTTPRoute (Gateway API)
For clusters using Istio, Envoy Gateway, Cilium, or other Gateway API implementations:
```yaml
# httproute-values.yaml
route:
enabled: true
hostnames:
- backup.example.com
parentRefs:
- name: my-gateway
namespace: istio-system
```
```bash
helm install postgresus oci://ghcr.io/rostislavdugin/charts/postgresus \
-n postgresus --create-namespace \
-f httproute-values.yaml
```
## Ingress Configuration
| Parameter | Description | Default Value |
| ----------------------- | ----------------- | ------------------------ |
| `ingress.enabled` | Enable Ingress | `false` |
| `ingress.className` | Ingress class | `nginx` |
| `ingress.hosts[0].host` | Hostname | `postgresus.example.com` |
| `ingress.tls` | TLS configuration | `[]` |
## HTTPRoute Configuration
| Parameter | Description | Default Value |
| ------------------ | ----------------------- | ------------------------------ |
| `route.enabled` | Enable HTTPRoute | `false` |
| `route.apiVersion` | Gateway API version | `gateway.networking.k8s.io/v1` |
| `route.hostnames` | Hostnames for the route | `["postgresus.example.com"]` |
| `route.parentRefs` | Gateway references | `[]` |
## Health Checks
| Parameter | Description | Default Value |
| ------------------------ | ---------------------- | ------------- |
| `livenessProbe.enabled` | Enable liveness probe | `true` |
| `readinessProbe.enabled` | Enable readiness probe | `true` |
## Custom Storage Size
```yaml
# storage-values.yaml
persistence:
size: 50Gi
storageClassName: "fast-ssd"
```
```bash
helm install postgresus oci://ghcr.io/rostislavdugin/charts/postgresus \
-n postgresus --create-namespace \
-f storage-values.yaml
```
## Upgrade
```bash
helm upgrade postgresus oci://ghcr.io/rostislavdugin/charts/postgresus -n postgresus
```
## Uninstall
```bash
helm uninstall postgresus -n postgresus
```

View File

@@ -0,0 +1,68 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "postgresus.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
*/}}
{{- define "postgresus.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "postgresus.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "postgresus.labels" -}}
helm.sh/chart: {{ include "postgresus.chart" . }}
{{ include "postgresus.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "postgresus.selectorLabels" -}}
app.kubernetes.io/name: {{ include "postgresus.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app: postgresus
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "postgresus.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "postgresus.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
{{/*
Namespace - uses the release namespace from helm install -n <namespace>
*/}}
{{- define "postgresus.namespace" -}}
{{- .Release.Namespace }}
{{- end }}

View File

@@ -0,0 +1,35 @@
{{- if .Values.route.enabled -}}
apiVersion: {{ .Values.route.apiVersion}}
kind: {{ .Values.route.kind}}
metadata:
name: {{ template "postgresus.fullname" . }}
annotations: {{ toYaml .Values.route.annotations | nindent 4 }}
labels:
app.kubernetes.io/component: "app"
{{- include "postgresus.labels" . | nindent 4 }}
spec:
{{- with .Values.route.parentRefs }}
parentRefs:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- with .Values.route.hostnames }}
hostnames:
{{- toYaml . | nindent 4 }}
{{- end }}
rules:
- backendRefs:
- name: {{ template "postgresus.fullname" . }}-service
port: {{ .Values.service.port }}
{{- with .Values.route.filters }}
filters:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.route.matches }}
matches:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.route.timeouts }}
timeouts:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}

View File

@@ -0,0 +1,42 @@
{{- if .Values.ingress.enabled -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ include "postgresus.fullname" . }}-ingress
namespace: {{ include "postgresus.namespace" . }}
labels:
{{- include "postgresus.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- if .Values.ingress.className }}
ingressClassName: {{ .Values.ingress.className }}
{{- end }}
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
pathType: {{ .pathType }}
backend:
service:
name: {{ include "postgresus.fullname" $ }}-service
port:
number: {{ $.Values.service.port }}
{{- end }}
{{- end }}
{{- end }}

View File

@@ -0,0 +1,36 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "postgresus.fullname" . }}-service
namespace: {{ include "postgresus.namespace" . }}
labels:
{{- include "postgresus.labels" . | nindent 4 }}
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: {{ .Values.service.targetPort }}
protocol: TCP
name: http
selector:
{{- include "postgresus.selectorLabels" . | nindent 4 }}
---
{{- if .Values.service.headless.enabled }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "postgresus.fullname" . }}-headless
namespace: {{ include "postgresus.namespace" . }}
labels:
{{- include "postgresus.labels" . | nindent 4 }}
spec:
type: ClusterIP
clusterIP: None
ports:
- port: {{ .Values.service.port }}
targetPort: {{ .Values.service.targetPort }}
protocol: TCP
name: http
selector:
{{- include "postgresus.selectorLabels" . | nindent 4 }}
{{- end }}

View File

@@ -0,0 +1,101 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ include "postgresus.fullname" . }}
namespace: {{ include "postgresus.namespace" . }}
labels:
{{- include "postgresus.labels" . | nindent 4 }}
spec:
serviceName: {{ include "postgresus.fullname" . }}-headless
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
{{- include "postgresus.selectorLabels" . | nindent 6 }}
template:
metadata:
annotations:
{{- with .Values.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "postgresus.selectorLabels" . | nindent 8 }}
{{- with .Values.podLabels }}
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
{{- if .Values.customRootCA }}
env:
- name: SSL_CERT_FILE
value: /etc/ssl/certs/custom-root-ca.crt
{{- end }}
ports:
- name: http
containerPort: {{ .Values.service.targetPort }}
protocol: TCP
volumeMounts:
- name: postgresus-storage
mountPath: {{ .Values.persistence.mountPath }}
{{- if .Values.customRootCA }}
- name: custom-root-ca
mountPath: /etc/ssl/certs/custom-root-ca.crt
subPath: ca.crt
readOnly: true
{{- end }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- if .Values.livenessProbe.enabled }}
livenessProbe:
httpGet:
{{- toYaml .Values.livenessProbe.httpGet | nindent 14 }}
initialDelaySeconds: {{ .Values.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.livenessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.livenessProbe.timeoutSeconds }}
failureThreshold: {{ .Values.livenessProbe.failureThreshold }}
{{- end }}
{{- if .Values.readinessProbe.enabled }}
readinessProbe:
httpGet:
{{- toYaml .Values.readinessProbe.httpGet | nindent 14 }}
initialDelaySeconds: {{ .Values.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.readinessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.readinessProbe.timeoutSeconds }}
failureThreshold: {{ .Values.readinessProbe.failureThreshold }}
{{- end }}
{{- if .Values.customRootCA }}
volumes:
- name: custom-root-ca
secret:
secretName: {{ .Values.customRootCA }}
{{- end }}
{{- if .Values.persistence.enabled }}
volumeClaimTemplates:
- metadata:
name: postgresus-storage
spec:
accessModes:
- {{ .Values.persistence.accessMode }}
{{- if .Values.persistence.storageClassName }}
storageClassName: {{ .Values.persistence.storageClassName }}
{{- end }}
resources:
requests:
storage: {{ .Values.persistence.size }}
{{- end }}
updateStrategy:
{{- toYaml .Values.updateStrategy | nindent 4 }}

104
deploy/helm/values.yaml Normal file
View File

@@ -0,0 +1,104 @@
# Default values for postgresus
# Image configuration
image:
repository: rostislavdugin/postgresus
tag: null
pullPolicy: Always
# StatefulSet configuration
replicaCount: 1
# RootCA setup, need name of secret in same namespace
customRootCA: ""
# Service configuration
service:
type: ClusterIP
port: 4005 # Service port
targetPort: 4005 # Internal container port
# Headless service for StatefulSet
headless:
enabled: true
# Resource limits and requests
resources:
requests:
memory: "1Gi"
cpu: "500m"
limits:
memory: "1Gi"
cpu: "500m"
# Persistent storage configuration
persistence:
enabled: true
# Storage class name. Leave empty to use cluster default.
# Examples: "longhorn", "standard", "gp2", etc.
storageClassName: ""
accessMode: ReadWriteOnce
size: 10Gi
# Mount path in container
mountPath: /postgresus-data
# Ingress configuration (disabled by default - using LoadBalancer instead)
ingress:
enabled: false
className: nginx
annotations: {}
hosts:
- host: postgresus.example.com
paths:
- path: /
pathType: Prefix
tls: []
# HTTPRoute configuration for Gateway API
route:
enabled: false
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
annotations: {}
hostnames:
- postgresus.example.com
parentRefs: []
filters: []
matches: []
timeouts: {}
# Health checks configuration
# Note: The application only has /api/v1/system/health endpoint
livenessProbe:
enabled: true
httpGet:
path: /api/v1/system/health
port: 4005
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
enabled: true
httpGet:
path: /api/v1/system/health
port: 4005
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
# StatefulSet update strategy
updateStrategy:
type: RollingUpdate
rollingUpdate:
partition: 0
# Pod labels and annotations
podLabels: {}
podAnnotations: {}
# Node selector, tolerations and affinity
nodeSelector: {}
tolerations: []
affinity: {}

View File

@@ -3,7 +3,10 @@
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/logo.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta
name="viewport"
content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"
/>
<meta name="robots" content="noindex" />
<title>Postgresus - PostgreSQL backups</title>

File diff suppressed because it is too large Load Diff

View File

@@ -8,7 +8,9 @@
"build": "tsc -b && vite build",
"lint": "eslint .",
"format": "prettier --write \"**/*.{ts,tsx,js,jsx,json,css,md}\"",
"preview": "vite preview"
"preview": "vite preview",
"test": "vitest run",
"test:watch": "vitest"
},
"dependencies": {
"@tailwindcss/vite": "^4.1.7",
@@ -22,6 +24,7 @@
"tailwindcss": "^4.1.7"
},
"devDependencies": {
"@vitest/coverage-v8": "^3.2.4",
"@eslint/js": "^9.25.0",
"@trivago/prettier-plugin-sort-imports": "^5.2.2",
"@types/react": "^19.1.2",
@@ -36,6 +39,7 @@
"prettier-plugin-tailwindcss": "^0.6.11",
"typescript": "~5.8.3",
"typescript-eslint": "^8.30.1",
"vite": "^6.3.5"
"vite": "^6.3.5",
"vitest": "^3.2.4"
}
}

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

After

Width:  |  Height:  |  Size: 32 KiB

View File

@@ -1,24 +1,12 @@
<svg width="1000" height="1000" viewBox="0 0 1000 1000" fill="none" xmlns="http://www.w3.org/2000/svg">
<g clip-path="url(#clip0_221_160)">
<path d="M213 149H388L725.5 233.5L856.5 602.5L811.5 685L733 650L679.5 631.5L581 591.5L492.5 570.5L430.5 545.5L371.5 494.5L166.5 386.5L213 149Z" fill="#155DFC"/>
<path d="M66.5436 450.256C242.608 569.316 275.646 627.237 321.721 606.141C333.899 600.56 366.664 580.46 376.871 491.549C373.963 491.413 368.225 491.432 361.922 493.872C322.424 509.249 323.556 596.384 303.865 596.481C303.553 596.481 302.87 596.481 301.992 596.228C300.06 595.876 297.679 595.135 294.108 593.476C283.667 588.597 278.437 586.138 270.514 581.357C270.514 581.357 262.415 576.537 255.039 571.385C244.149 563.774 237.924 554.602 225.708 543.635C215.482 534.463 215.345 536.863 191.478 519.339C183.263 513.308 177.233 508.547 165.504 499.258C146.243 484.016 132.934 473.478 121.576 463.662C107.369 451.387 88.1466 433.258 66.8558 408.103C47.4969 371.708 24.4303 333.868 12.7213 337.674C5.24705 340.113 4.09568 358.535 5.53979 370.888C10.36 412.553 50.6584 439.542 66.5436 450.275V450.256Z" fill="#003A86"/>
<path d="M86.9766 439.249C181.683 457.672 276.369 476.113 371.076 494.535C382.082 511.572 393.752 522.169 401.734 528.394C401.734 528.394 448.823 565.101 520.326 562.369C523.761 562.233 532.289 561.569 543.881 562.818C555.785 564.106 557.853 565.375 576.763 568.79C596.2 572.302 598.386 571.756 606.153 574.761C615.188 578.255 621.589 582.626 626.078 585.729C633.513 590.861 635.269 593.359 649.496 605.654C657.38 612.464 661.322 615.879 664.932 618.611C671.567 623.627 682.203 631.608 697.327 638.048C705.933 641.698 714.402 644.196 722.443 646.557C726.853 647.845 730.561 648.84 733.195 649.504C734.522 662.227 735.459 680.337 733.195 701.823C730.834 724.285 729.195 739.936 719.75 757.636C715.554 765.501 700.313 792.978 667.43 804.979C660.795 807.399 640.441 814.503 616.106 807.965C607.929 805.78 592.746 801.467 580.725 787.045C568.391 772.233 567.084 756.016 566.772 751.177C566.167 741.868 565.328 729.008 574.246 718.294C581.291 709.844 593.039 704.38 604.143 705.336C617.433 706.487 621.101 716.128 630.371 713.708C634.801 712.557 634.762 710.156 646.978 697.374C657.985 685.86 660.541 685.158 660.931 681.138C661.849 671.497 648.383 662.93 642.997 659.495C638.977 656.939 629.142 651.299 611.598 649.035C601.586 647.747 582.169 645.464 559.785 654.519C532.484 665.545 518.804 686.114 514.452 692.885C503.016 710.605 500.245 727.212 498.508 738.219C497.552 744.327 494.937 763.666 500.011 788.04C507.114 822.231 524.756 844.595 530.396 851.328C554.868 880.522 584.335 891.314 598.152 896.173C662.766 918.908 720.959 895.061 732.181 890.201C793.848 863.524 821.325 811.575 834.829 786.069C860.492 737.575 863.75 693.959 867.224 647.552C868.395 632.018 871.03 586.314 861.253 527.964C854.286 486.476 844.138 457.086 833.346 425.823C810.826 360.585 786.022 313.944 778.041 299.269C778.041 299.269 746.075 240.548 692.838 177.69C682.456 165.435 671.996 153.901 654.979 144.3C644.129 138.192 624.614 129.469 574.265 128.356C548.798 127.79 512.735 129.098 469.626 137.821C471.792 141.139 473.939 144.456 476.105 147.793C464.24 141.568 446.969 133.333 425.288 125.878C403.9 118.521 387.702 115.164 362.001 109.934C321.82 101.757 291.572 98.2838 284.273 97.4837C251.644 93.8539 227.758 93.1318 203.559 94.9858C188.259 96.1567 174.267 98.2447 159.709 106.441C150.459 111.652 144.019 117.623 136.291 124.883C121.342 138.914 111.878 151.657 107.389 157.765C75.5994 201.011 59.7142 222.653 44.6096 247.944C44.6096 247.944 33.6227 266.347 15.22 306.743C12.2538 313.241 7.12134 324.775 5.87238 340.64C5.67723 343.08 5.05274 351.003 6.26267 358.574C9.32652 377.816 23.5724 390.911 38.1501 403.42C70.3693 431.073 92.9677 446.763 92.9677 446.763C96.617 449.202 131.92 473.908 162.226 497.58C211.697 536.239 236.422 555.578 251.566 569.219C251.566 569.219 264.192 580.596 289.757 592.13C297.367 595.564 303.261 597.594 310.189 596.482C320.22 594.862 326.523 587.544 332.631 580.284C334.836 577.65 339.383 571.893 353.922 539.381C357.532 531.321 359.328 527.281 361.513 521.817C363.328 517.29 367.778 505.893 372.656 489.325C376.501 476.308 384.112 450.51 388.424 421.823C392.347 395.77 398.026 358.028 387.917 310.704C387 306.45 376.95 261.117 348.731 218.359C334.036 196.093 319.712 182.53 316.668 179.661C307.457 171.055 298.851 164.674 292.255 160.224C298.675 160.537 307.476 161.395 317.663 163.718C325.176 165.415 345.374 170.626 367.485 185.145C389.107 199.332 402.065 215.549 407.842 223.511C428.254 251.632 433.66 279.539 436.997 296.751C440.9 316.91 441.915 337.752 442.735 354.047C443.691 373.64 443.008 379.592 445.232 393.916C446.755 403.752 449.409 420.164 458.015 439.249C470.953 467.936 489.531 485.402 496.537 491.569C509.007 502.517 520.131 508.157 542.378 519.475C560.995 528.94 573.134 532.98 586.716 534.424C592.668 535.048 597.683 535.048 601.157 534.931C597.722 526.794 595.576 519.515 594.19 513.758C593 508.84 591.165 501.034 590.697 490.593C590.326 482.358 590.951 475.372 591.692 470.161C593.156 476.504 595.342 484.719 598.659 494.086C600.259 498.594 606.036 514.441 618.096 534.951C627.053 550.212 637.904 563.618 655.955 582.294C673.167 600.072 698.517 623.959 733.176 649.562C623.326 589.359 513.496 529.155 403.646 468.951" fill="#155DFC"/>
<path d="M562.652 325.145C573.171 318.588 582.07 313.885 588.568 310.704C594.501 307.777 597.565 306.645 601.194 305.728C603.712 305.084 611.811 303.249 622.29 304.674C626.642 305.279 632.77 306.177 639.736 310.041C648.069 314.666 652.577 320.774 653.69 322.335C655.017 324.209 657.905 328.326 659.661 334.298C660.754 337.947 660.52 339.313 661.984 342.611C663.525 346.065 665.223 347.9 667.955 351.256C670.16 353.949 673.166 357.852 676.6 362.887C674.785 361.872 665.594 356.896 656.012 357.072C651.68 357.15 647.367 358.243 647.367 358.243C643.893 359.121 641.571 360.214 640.732 360.565C632.984 363.785 615.284 359.511 608.181 347.939C603.692 340.64 606.834 335.918 601.507 331.156C599.536 329.4 596.94 328.092 585.27 326.687C579.748 326.023 572.039 325.321 562.691 325.184L562.652 325.145Z" fill="#003C8D"/>
<path d="M820.797 391.692C811.059 390.95 795.467 391.009 777.572 396.726C763.833 401.117 753.373 407.343 746.348 412.338C752.319 411.031 760.086 409.509 769.258 408.182C771.444 407.87 778.489 406.874 788.363 406.016C797.906 405.196 810.552 404.435 825.715 404.513C824.076 400.239 822.437 395.965 820.797 391.672V391.692Z" fill="#0052C9"/>
<path d="M841.795 450.627C830.925 450.256 815.625 450.997 798.218 456.032C786.177 459.506 776.283 464.15 768.711 468.405C774.019 467.624 780.693 466.765 788.401 466.043C790.977 465.809 798.549 465.107 808.834 464.677C818.455 464.267 831.12 464.033 846.244 464.599C844.761 459.955 843.278 455.291 841.814 450.646L841.795 450.627Z" fill="#0052C9"/>
<path d="M855.848 500.39C845.895 499.824 829.952 500.331 812.583 507.65C805.694 510.557 799.938 513.953 795.273 517.231C798.415 516.607 802.748 515.846 807.919 515.143C818.808 513.699 827.141 513.445 835.494 513.231C841.524 513.075 849.369 513.016 858.6 513.328L855.848 500.37V500.39Z" fill="#0052C9"/>
<path d="M570.282 597.028C599.399 605.985 630.096 616.016 657.475 624.934C674.434 630.457 688.289 635.043 697.344 638.048C665.281 603.663 633.238 569.297 601.174 534.912C596.471 534.853 589.095 534.502 580.254 532.824C573.971 531.633 559.627 528.394 529.769 512.157C516.031 504.683 509.161 500.956 502.526 496.038C469.975 471.917 457.427 437.142 453.836 426.799C450.656 417.607 448.47 397.116 444.235 356.525C438.732 303.893 439.552 300.537 435.766 285.276C432.819 273.392 426.203 245.329 407.859 219.511C401.81 210.983 378.099 178.607 333.858 164.927C315.378 159.209 299.141 158.565 288.271 158.956C297.034 165.298 309.016 174.86 321.662 188.111C327.926 194.668 354.408 223.16 373.493 269.859C379.465 284.457 389.866 310.489 392.423 346.085C394.316 372.371 391.154 392.94 390.432 397.409C385.671 426.877 376.264 444.87 371.854 486.436C371.483 489.91 371.23 492.759 371.093 494.555C377.318 504.059 383.368 511.142 387.973 516.06C392.442 520.822 396.092 523.964 397.224 525.076C397.224 525.076 403.605 530.716 410.923 535.614C444.43 558.095 570.282 597.028 570.282 597.028Z" fill="#0051C8"/>
<path d="M591.456 468.893C590.109 479.079 589.289 494.594 593.427 512.626C595.495 521.602 598.013 527.886 600.881 534.912C607.555 551.304 615.908 571.619 634.018 591.915C647.424 606.941 661.007 616.035 671.916 623.393C676.56 626.515 682.786 630.691 691.47 635.16C708.175 643.786 723.24 648.001 733.759 650.226C732.139 643.434 730.5 636.663 728.88 629.872C712.097 621.968 691.86 610.435 671.155 593.476C663.622 587.29 658.177 582.157 655.64 579.718C646.019 570.448 627.792 552.729 612.512 524.881C600.53 503.044 594.597 483.021 591.456 468.854V468.893Z" fill="#00398B"/>
<path d="M680.074 799.203C689.988 795.768 704.312 789.367 718.519 777.307C731.438 766.34 738.912 755.236 741.937 750.396C750.816 736.248 754.251 723.836 756.631 715.015C759.305 705.121 762.154 690.485 761.861 672.414C765.569 674.502 770.819 677.215 777.298 679.888C791.524 685.743 803.175 687.85 810.434 689.099C824.914 691.617 836.272 691.773 845.054 691.831C852.469 691.89 858.695 691.617 863.222 691.324C863.749 686.64 864.257 681.898 864.725 677.117C865.72 667.047 866.54 657.153 867.223 647.474C861.134 647.396 854.733 647.142 848.039 646.674C842.927 646.303 837.989 645.815 833.267 645.269C826.846 646.147 818.24 647.006 808.053 647.064C778.586 647.201 757.178 640.507 751.87 638.77C741.917 635.511 733.955 631.862 728.452 629.052C729.428 632.682 730.696 637.853 731.808 644.117C733.135 651.553 736.238 671.458 733.174 701.784C731.125 722.06 730.091 732.423 725.368 744.854C721.309 755.567 710.322 778.888 680.035 799.144L680.074 799.203Z" fill="#0050C8"/>
<path d="M493.94 571.483C489.179 567.034 474.133 552.846 461.233 537.625C458.15 533.995 453.017 527.867 446.89 519.183C444.86 516.295 442.733 513.153 440.547 509.718C435.395 501.62 428.682 490.925 422.789 475.626C419.139 466.141 416.934 457.847 415.568 451.7C411.88 469.225 412.465 483.783 413.578 493.306C414.846 504.274 417.246 512.255 419.315 518.968C421.54 526.208 424.857 535.497 429.931 545.918C436.937 549.49 442.967 552.28 447.553 554.31C460.453 560.028 470.756 563.677 477.45 566.019C484.144 568.38 489.823 570.195 493.96 571.483H493.94Z" fill="#003C8D"/>
<path d="M668.582 578.488C670.768 569.18 674.593 562.798 677.13 559.227C684.135 549.333 692.507 545.586 701.855 541.293C709.407 537.819 715.437 535.165 720.999 537.78C722.014 538.248 722.482 538.658 728.337 542.425C730.366 543.771 733.333 545.703 736.982 547.908C746.525 553.685 784.618 575.854 840.177 582.294C877.822 586.665 906.762 581.864 923.467 578.957C957.599 573.024 985.271 562.545 1005 553.392C994.15 582.001 979.26 598.999 970.186 607.8C935.371 641.58 892.731 644.976 865.839 647.591C859.79 648.176 838.518 649.757 809.344 645.269C745.822 635.511 697.503 602.824 668.582 578.508V578.488Z" fill="#8BC7FE"/>
<path d="M664.369 591.27C671.004 599.311 678.791 607.839 687.826 616.425C700.764 628.72 713.391 638.38 724.631 645.912C724.475 642.595 723.89 638.341 722.172 633.794C721.665 632.467 721.119 631.257 720.572 630.144C726.017 633.15 732.008 636.018 738.487 638.536C769.477 650.537 798.183 649.386 817.874 646.225C791.353 642.653 756.792 634.262 720.143 614.318C707.361 607.351 695.964 599.877 685.953 592.441C685.484 592.09 681.874 589.202 676.8 585.221C673.404 582.567 670.614 580.381 668.779 578.957C667.296 583.074 665.813 587.172 664.33 591.29L664.369 591.27Z" fill="#00398B"/>
<path d="M992.766 578.937C975.417 607.683 953.073 622.065 944.135 627.237C936.231 631.803 924.366 637.56 908.91 641.092C908.715 641.151 888.81 646.4 853.956 648.235C848.863 648.508 843.887 648.567 843.887 648.567C843.887 648.567 837.759 648.645 831.436 648.391C776.54 646.283 723.362 623.88 723.362 623.88C715.146 620.641 701.642 614.864 687.747 606.434C675.511 599.018 668.74 593.066 665.149 589.709C663.432 588.09 662.046 586.704 661.109 585.728C668.447 572.77 675.785 559.832 683.142 546.874C690.011 553.743 700.198 563.228 713.468 573.024C732.007 586.704 785.107 625.949 857.059 623.587C937.5 620.972 991.069 568.028 1005 553.333C1002.6 559.851 998.777 568.926 992.747 578.898L992.766 578.937Z" fill="#0087F7"/>
<path d="M649.906 574.117C651.233 571.268 653.146 567.423 655.761 563.111C656.971 561.12 661.323 553.997 666.533 548.416C670.67 543.986 682.047 531.789 698.733 531.282C708.119 530.989 715.847 534.502 721.019 537.761C717.604 538.073 712.94 538.736 707.534 540.239C701.796 541.859 695.279 543.323 689.502 547.909C681.462 554.29 677.013 562.896 668.582 580.791C666.884 584.421 665.577 587.426 664.757 589.358C661.596 586.294 658.415 583.094 655.195 579.737C653.38 577.845 651.624 575.971 649.906 574.098V574.117Z" fill="#0051CB"/>
<svg width="32" height="32" viewBox="0 0 32 32" fill="none" xmlns="http://www.w3.org/2000/svg">
<g clip-path="url(#clip0_286_152)">
<path d="M12.5378 28.7972C12.5378 30.2972 14.2889 30.2982 14.7498 29.4999C15.0384 28.9999 15.2498 28.4999 15.2498 26.9999C15.2498 25.4999 14.5258 24.2384 13.8296 22.8794C13.6504 22.5296 13.4578 22.1537 13.253 21.7309C12.9016 21.0056 12.4714 20.3367 11.8469 19.8035C11.7511 19.7217 11.7231 19.5023 11.7404 19.3555C11.8085 18.7759 11.8827 18.1969 11.9568 17.6179C12.0794 16.6609 12.202 15.7039 12.2975 14.7442C12.3645 14.0718 12.3633 13.378 12.2786 12.7082C12.0956 11.2586 11.2821 10.1946 10.0335 9.47786C9.6702 9.26926 9.56307 9.03637 9.60844 8.64632C9.73481 7.56183 10.1635 6.58265 10.7083 5.66605C11.2087 4.82425 11.7757 4.02144 12.3417 3.2201C12.4142 3.11747 12.4866 3.01486 12.5589 2.9122C12.6515 2.78077 12.8805 2.68113 13.049 2.67687C15.103 2.62543 17.1573 2.59121 19.2117 2.56589C19.3528 2.56407 19.5394 2.62989 19.6303 2.73115C20.9118 4.1585 22.0569 5.678 22.7442 7.49459C22.7912 7.61894 22.8303 7.74673 22.8661 7.87492C23.1241 8.79206 23.1199 8.79508 22.3848 9.31247L22.3515 9.33589C20.9673 10.3108 20.4612 11.705 20.5228 13.3373C20.5933 15.2138 21.0153 17.0328 21.4965 18.8385C21.5943 19.2054 21.6577 19.4768 21.3049 19.8152C20.5985 20.4924 20.3047 21.4218 20.1641 22.3672C20.0179 23.352 19.8995 24.3409 19.7811 25.3299C19.7154 25.8788 19.7122 26.3421 19.5792 26.9759C19.0702 29.4029 17.7498 31.9999 15.7498 31.9999H12.5378C11.2498 31.9999 10.2498 30.7972 10.2498 28.7972H12.5378Z" fill="#155DFC"/>
<path d="M11.5607 1.64008C10.8347 2.775 10.091 3.85078 9.44034 4.98022C8.85324 5.9991 8.46218 7.10566 8.27283 8.28027C8.02494 9.8182 8.4235 11.1797 9.51913 12.2444C10.4211 13.1209 10.6538 14.1118 10.538 15.2919C10.2857 17.8647 9.77936 20.385 9.05131 22.863C9.04422 22.8869 9.02214 22.9064 8.97435 22.9744C8.53792 22.8381 8.07901 22.7111 7.63165 22.5524C1.38253 20.3344 -3.18062 16.2382 -6.04606 10.2575C-6.29374 9.74063 -6.51975 9.21246 -6.73158 8.68004C-6.77189 8.57878 -6.74374 8.38234 -6.66961 8.31328C-4.28698 6.09307 -1.91002 3.86617 0.50117 1.67735C2.16142 0.170403 4.14529 -0.303895 6.31143 0.184986C7.98444 0.562481 9.62828 1.06959 11.2841 1.52262C11.3961 1.5532 11.5 1.61376 11.5607 1.64008Z" fill="#155DFC"/>
<path d="M24.2397 22.3314C23.8972 21.1864 23.5199 20.1003 23.2528 18.9878C22.9016 17.5244 22.6171 16.0438 22.3435 14.5632C22.1888 13.7264 22.35 12.9544 22.9725 12.3027C24.5667 10.6339 24.5983 8.69929 23.8328 6.67592C23.1868 4.96849 22.1482 3.49759 20.9256 2.15226C20.7825 1.9947 20.6423 1.8341 20.4795 1.65123C20.5718 1.6006 20.6427 1.54693 20.7227 1.52C22.532 0.910212 24.3624 0.387106 26.2602 0.122211C28.1953 -0.147949 29.8448 0.454544 31.2612 1.74398C32.5042 2.87545 33.7011 4.05816 34.9529 5.17971C35.9556 6.07788 37.0032 6.92664 38.0476 7.77681C38.2361 7.93012 38.2841 8.05386 38.2282 8.27602C37.2646 12.1148 35.328 15.3721 32.3643 17.9969C30.0283 20.0657 27.3376 21.4946 24.2397 22.3316V22.3314Z" fill="#155DFC"/>
</g>
<defs>
<clipPath id="clip0_221_160">
<rect width="1000" height="1000" fill="white"/>
<clipPath id="clip0_286_152">
<rect width="32" height="32" rx="6" fill="white"/>
</clipPath>
</defs>
</svg>

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 3.0 KiB

View File

@@ -0,0 +1,3 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
<svg width="800px" height="800px" viewBox="0 0 1024 1024" class="icon" version="1.1" xmlns="http://www.w3.org/2000/svg"><path d="M853.333333 256H469.333333l-85.333333-85.333333H170.666667c-46.933333 0-85.333333 38.4-85.333334 85.333333v170.666667h853.333334v-85.333334c0-46.933333-38.4-85.333333-85.333334-85.333333z" fill="#FFA000" /><path d="M853.333333 256H170.666667c-46.933333 0-85.333333 38.4-85.333334 85.333333v426.666667c0 46.933333 38.4 85.333333 85.333334 85.333333h682.666666c46.933333 0 85.333333-38.4 85.333334-85.333333V341.333333c0-46.933333-38.4-85.333333-85.333334-85.333333z" fill="#FFCA28" /></svg>

After

Width:  |  Height:  |  Size: 741 B

View File

@@ -0,0 +1,102 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="64"
height="64"
viewBox="0 0 64 64"
version="1.1"
xml:space="preserve"
style="clip-rule:evenodd;fill-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:1.41420996"
id="svg50"
sodipodi:docname="rclone-icon.svg"
inkscape:version="0.92.4 (5da689c313, 2019-01-14)"><metadata
id="metadata56"><rdf:RDF><cc:Work
rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" /><dc:title></dc:title></cc:Work></rdf:RDF></metadata><defs
id="defs54">
<clipPath
id="_clip1">
<rect
x="14"
y="579"
width="257"
height="84"
id="rect4" />
</clipPath>
</defs><sodipodi:namedview
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1"
objecttolerance="10"
gridtolerance="10"
guidetolerance="10"
inkscape:pageopacity="0"
inkscape:pageshadow="2"
inkscape:window-width="1531"
inkscape:window-height="807"
id="namedview52"
showgrid="false"
units="px"
inkscape:zoom="1.539823"
inkscape:cx="-84.425288"
inkscape:cy="26.5"
inkscape:window-x="70"
inkscape:window-y="27"
inkscape:window-maximized="0"
inkscape:current-layer="svg50" />
<g
id="g824"
transform="matrix(1.3422256,0,0,1.3422256,-2.2309418e-8,3.8420351)"><path
d="m 45.726917,21.83581 c -1.507672,-2.611426 -3.701518,-4.579735 -6.222732,-5.808561 -0.322585,1.72227 -0.932898,3.419936 -1.857594,5.021921 l -1.459147,2.532147 c 0.971853,0.539918 1.817954,1.334759 2.414598,2.368122 1.753027,3.035842 0.712146,6.919151 -2.324383,8.672176 -3.035847,1.753025 -6.919159,0.712829 -8.672186,-2.323698 l -2.944264,-5.091631 h -4.751283 l -2.375642,4.114312 2.946315,5.090948 c 4.025469,6.971776 12.939592,9.360401 19.911375,5.334937 6.971101,-4.024782 9.359727,-12.938896 5.334943,-19.910673"
style="fill:#70caf2;fill-rule:nonzero;stroke-width:0.68344086"
id="path7"
inkscape:connector-curvature="0" /><path
d="M 31.127807,0.45456543 C 24.156023,-3.5702158 15.2419,-1.1815912 11.217114,5.7895021 9.7087599,8.4009285 9.1018638,11.285048 9.2980112,14.083052 10.950572,13.501445 12.726153,13.180911 14.576228,13.180911 l 2.921711,-0.0027 c -0.01845,-1.111274 0.247406,-2.241684 0.843367,-3.2743635 1.75371,-3.036526 5.636339,-4.0774059 8.672868,-2.3236971 3.03653,1.7530242 4.076727,5.6356506 2.323701,8.6721766 l -2.936747,5.095732 2.374958,4.114995 4.751283,-6.83e-4 2.93538,-5.097099 C 40.488218,13.394145 38.099591,4.4793466 31.127807,0.45456543"
style="fill:#b4e3f9;fill-rule:nonzero;stroke-width:0.68344086"
id="path11"
inkscape:connector-curvature="0" /><path
d="m 19.297646,37.095505 -1.463932,-2.529413 c -0.9534,0.57204 -2.064675,0.906925 -3.25728,0.906925 -3.506736,0 -6.3491688,-2.842428 -6.3491688,-6.349162 0,-3.50605 2.8424328,-6.348479 6.3491688,-6.348479 l 5.881011,-0.0041 2.376326,-4.114312 -2.376326,-4.114312 -5.881695,0.0055 C 6.5254965,14.548074 1.6621211e-8,21.074248 1.6621211e-8,29.12381 1.6621211e-8,37.174056 6.5254965,43.70023 14.57575,43.70023 c 3.014659,0 5.814718,-0.915811 8.139101,-2.48294 -1.329976,-1.140662 -2.49251,-2.520528 -3.417205,-4.12183"
style="fill:#3f79ad;fill-rule:nonzero;stroke-width:0.68344086"
id="path15"
inkscape:connector-curvature="0" /></g>
</svg>

After

Width:  |  Height:  |  Size: 4.3 KiB

View File

@@ -0,0 +1,3 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
<svg width="800px" height="800px" viewBox="0 0 1024 1024" class="icon" version="1.1" xmlns="http://www.w3.org/2000/svg"><path d="M853.333333 256H469.333333l-85.333333-85.333333H170.666667c-46.933333 0-85.333333 38.4-85.333334 85.333333v170.666667h853.333334v-85.333334c0-46.933333-38.4-85.333333-85.333334-85.333333z" fill="#FFA000" /><path d="M853.333333 256H170.666667c-46.933333 0-85.333333 38.4-85.333334 85.333333v426.666667c0 46.933333 38.4 85.333333 85.333334 85.333333h682.666666c46.933333 0 85.333333-38.4 85.333334-85.333333V341.333333c0-46.933333-38.4-85.333333-85.333334-85.333333z" fill="#FFCA28" /></svg>

After

Width:  |  Height:  |  Size: 741 B

View File

@@ -1,24 +1,12 @@
<svg width="1000" height="1000" viewBox="0 0 1000 1000" fill="none" xmlns="http://www.w3.org/2000/svg">
<g clip-path="url(#clip0_221_160)">
<path d="M213 149H388L725.5 233.5L856.5 602.5L811.5 685L733 650L679.5 631.5L581 591.5L492.5 570.5L430.5 545.5L371.5 494.5L166.5 386.5L213 149Z" fill="#155DFC"/>
<path d="M66.5436 450.256C242.608 569.316 275.646 627.237 321.721 606.141C333.899 600.56 366.664 580.46 376.871 491.549C373.963 491.413 368.225 491.432 361.922 493.872C322.424 509.249 323.556 596.384 303.865 596.481C303.553 596.481 302.87 596.481 301.992 596.228C300.06 595.876 297.679 595.135 294.108 593.476C283.667 588.597 278.437 586.138 270.514 581.357C270.514 581.357 262.415 576.537 255.039 571.385C244.149 563.774 237.924 554.602 225.708 543.635C215.482 534.463 215.345 536.863 191.478 519.339C183.263 513.308 177.233 508.547 165.504 499.258C146.243 484.016 132.934 473.478 121.576 463.662C107.369 451.387 88.1466 433.258 66.8558 408.103C47.4969 371.708 24.4303 333.868 12.7213 337.674C5.24705 340.113 4.09568 358.535 5.53979 370.888C10.36 412.553 50.6584 439.542 66.5436 450.275V450.256Z" fill="#003A86"/>
<path d="M86.9766 439.249C181.683 457.672 276.369 476.113 371.076 494.535C382.082 511.572 393.752 522.169 401.734 528.394C401.734 528.394 448.823 565.101 520.326 562.369C523.761 562.233 532.289 561.569 543.881 562.818C555.785 564.106 557.853 565.375 576.763 568.79C596.2 572.302 598.386 571.756 606.153 574.761C615.188 578.255 621.589 582.626 626.078 585.729C633.513 590.861 635.269 593.359 649.496 605.654C657.38 612.464 661.322 615.879 664.932 618.611C671.567 623.627 682.203 631.608 697.327 638.048C705.933 641.698 714.402 644.196 722.443 646.557C726.853 647.845 730.561 648.84 733.195 649.504C734.522 662.227 735.459 680.337 733.195 701.823C730.834 724.285 729.195 739.936 719.75 757.636C715.554 765.501 700.313 792.978 667.43 804.979C660.795 807.399 640.441 814.503 616.106 807.965C607.929 805.78 592.746 801.467 580.725 787.045C568.391 772.233 567.084 756.016 566.772 751.177C566.167 741.868 565.328 729.008 574.246 718.294C581.291 709.844 593.039 704.38 604.143 705.336C617.433 706.487 621.101 716.128 630.371 713.708C634.801 712.557 634.762 710.156 646.978 697.374C657.985 685.86 660.541 685.158 660.931 681.138C661.849 671.497 648.383 662.93 642.997 659.495C638.977 656.939 629.142 651.299 611.598 649.035C601.586 647.747 582.169 645.464 559.785 654.519C532.484 665.545 518.804 686.114 514.452 692.885C503.016 710.605 500.245 727.212 498.508 738.219C497.552 744.327 494.937 763.666 500.011 788.04C507.114 822.231 524.756 844.595 530.396 851.328C554.868 880.522 584.335 891.314 598.152 896.173C662.766 918.908 720.959 895.061 732.181 890.201C793.848 863.524 821.325 811.575 834.829 786.069C860.492 737.575 863.75 693.959 867.224 647.552C868.395 632.018 871.03 586.314 861.253 527.964C854.286 486.476 844.138 457.086 833.346 425.823C810.826 360.585 786.022 313.944 778.041 299.269C778.041 299.269 746.075 240.548 692.838 177.69C682.456 165.435 671.996 153.901 654.979 144.3C644.129 138.192 624.614 129.469 574.265 128.356C548.798 127.79 512.735 129.098 469.626 137.821C471.792 141.139 473.939 144.456 476.105 147.793C464.24 141.568 446.969 133.333 425.288 125.878C403.9 118.521 387.702 115.164 362.001 109.934C321.82 101.757 291.572 98.2838 284.273 97.4837C251.644 93.8539 227.758 93.1318 203.559 94.9858C188.259 96.1567 174.267 98.2447 159.709 106.441C150.459 111.652 144.019 117.623 136.291 124.883C121.342 138.914 111.878 151.657 107.389 157.765C75.5994 201.011 59.7142 222.653 44.6096 247.944C44.6096 247.944 33.6227 266.347 15.22 306.743C12.2538 313.241 7.12134 324.775 5.87238 340.64C5.67723 343.08 5.05274 351.003 6.26267 358.574C9.32652 377.816 23.5724 390.911 38.1501 403.42C70.3693 431.073 92.9677 446.763 92.9677 446.763C96.617 449.202 131.92 473.908 162.226 497.58C211.697 536.239 236.422 555.578 251.566 569.219C251.566 569.219 264.192 580.596 289.757 592.13C297.367 595.564 303.261 597.594 310.189 596.482C320.22 594.862 326.523 587.544 332.631 580.284C334.836 577.65 339.383 571.893 353.922 539.381C357.532 531.321 359.328 527.281 361.513 521.817C363.328 517.29 367.778 505.893 372.656 489.325C376.501 476.308 384.112 450.51 388.424 421.823C392.347 395.77 398.026 358.028 387.917 310.704C387 306.45 376.95 261.117 348.731 218.359C334.036 196.093 319.712 182.53 316.668 179.661C307.457 171.055 298.851 164.674 292.255 160.224C298.675 160.537 307.476 161.395 317.663 163.718C325.176 165.415 345.374 170.626 367.485 185.145C389.107 199.332 402.065 215.549 407.842 223.511C428.254 251.632 433.66 279.539 436.997 296.751C440.9 316.91 441.915 337.752 442.735 354.047C443.691 373.64 443.008 379.592 445.232 393.916C446.755 403.752 449.409 420.164 458.015 439.249C470.953 467.936 489.531 485.402 496.537 491.569C509.007 502.517 520.131 508.157 542.378 519.475C560.995 528.94 573.134 532.98 586.716 534.424C592.668 535.048 597.683 535.048 601.157 534.931C597.722 526.794 595.576 519.515 594.19 513.758C593 508.84 591.165 501.034 590.697 490.593C590.326 482.358 590.951 475.372 591.692 470.161C593.156 476.504 595.342 484.719 598.659 494.086C600.259 498.594 606.036 514.441 618.096 534.951C627.053 550.212 637.904 563.618 655.955 582.294C673.167 600.072 698.517 623.959 733.176 649.562C623.326 589.359 513.496 529.155 403.646 468.951" fill="#155DFC"/>
<path d="M562.652 325.145C573.171 318.588 582.07 313.885 588.568 310.704C594.501 307.777 597.565 306.645 601.194 305.728C603.712 305.084 611.811 303.249 622.29 304.674C626.642 305.279 632.77 306.177 639.736 310.041C648.069 314.666 652.577 320.774 653.69 322.335C655.017 324.209 657.905 328.326 659.661 334.298C660.754 337.947 660.52 339.313 661.984 342.611C663.525 346.065 665.223 347.9 667.955 351.256C670.16 353.949 673.166 357.852 676.6 362.887C674.785 361.872 665.594 356.896 656.012 357.072C651.68 357.15 647.367 358.243 647.367 358.243C643.893 359.121 641.571 360.214 640.732 360.565C632.984 363.785 615.284 359.511 608.181 347.939C603.692 340.64 606.834 335.918 601.507 331.156C599.536 329.4 596.94 328.092 585.27 326.687C579.748 326.023 572.039 325.321 562.691 325.184L562.652 325.145Z" fill="#003C8D"/>
<path d="M820.797 391.692C811.059 390.95 795.467 391.009 777.572 396.726C763.833 401.117 753.373 407.343 746.348 412.338C752.319 411.031 760.086 409.509 769.258 408.182C771.444 407.87 778.489 406.874 788.363 406.016C797.906 405.196 810.552 404.435 825.715 404.513C824.076 400.239 822.437 395.965 820.797 391.672V391.692Z" fill="#0052C9"/>
<path d="M841.795 450.627C830.925 450.256 815.625 450.997 798.218 456.032C786.177 459.506 776.283 464.15 768.711 468.405C774.019 467.624 780.693 466.765 788.401 466.043C790.977 465.809 798.549 465.107 808.834 464.677C818.455 464.267 831.12 464.033 846.244 464.599C844.761 459.955 843.278 455.291 841.814 450.646L841.795 450.627Z" fill="#0052C9"/>
<path d="M855.848 500.39C845.895 499.824 829.952 500.331 812.583 507.65C805.694 510.557 799.938 513.953 795.273 517.231C798.415 516.607 802.748 515.846 807.919 515.143C818.808 513.699 827.141 513.445 835.494 513.231C841.524 513.075 849.369 513.016 858.6 513.328L855.848 500.37V500.39Z" fill="#0052C9"/>
<path d="M570.282 597.028C599.399 605.985 630.096 616.016 657.475 624.934C674.434 630.457 688.289 635.043 697.344 638.048C665.281 603.663 633.238 569.297 601.174 534.912C596.471 534.853 589.095 534.502 580.254 532.824C573.971 531.633 559.627 528.394 529.769 512.157C516.031 504.683 509.161 500.956 502.526 496.038C469.975 471.917 457.427 437.142 453.836 426.799C450.656 417.607 448.47 397.116 444.235 356.525C438.732 303.893 439.552 300.537 435.766 285.276C432.819 273.392 426.203 245.329 407.859 219.511C401.81 210.983 378.099 178.607 333.858 164.927C315.378 159.209 299.141 158.565 288.271 158.956C297.034 165.298 309.016 174.86 321.662 188.111C327.926 194.668 354.408 223.16 373.493 269.859C379.465 284.457 389.866 310.489 392.423 346.085C394.316 372.371 391.154 392.94 390.432 397.409C385.671 426.877 376.264 444.87 371.854 486.436C371.483 489.91 371.23 492.759 371.093 494.555C377.318 504.059 383.368 511.142 387.973 516.06C392.442 520.822 396.092 523.964 397.224 525.076C397.224 525.076 403.605 530.716 410.923 535.614C444.43 558.095 570.282 597.028 570.282 597.028Z" fill="#0051C8"/>
<path d="M591.456 468.893C590.109 479.079 589.289 494.594 593.427 512.626C595.495 521.602 598.013 527.886 600.881 534.912C607.555 551.304 615.908 571.619 634.018 591.915C647.424 606.941 661.007 616.035 671.916 623.393C676.56 626.515 682.786 630.691 691.47 635.16C708.175 643.786 723.24 648.001 733.759 650.226C732.139 643.434 730.5 636.663 728.88 629.872C712.097 621.968 691.86 610.435 671.155 593.476C663.622 587.29 658.177 582.157 655.64 579.718C646.019 570.448 627.792 552.729 612.512 524.881C600.53 503.044 594.597 483.021 591.456 468.854V468.893Z" fill="#00398B"/>
<path d="M680.074 799.203C689.988 795.768 704.312 789.367 718.519 777.307C731.438 766.34 738.912 755.236 741.937 750.396C750.816 736.248 754.251 723.836 756.631 715.015C759.305 705.121 762.154 690.485 761.861 672.414C765.569 674.502 770.819 677.215 777.298 679.888C791.524 685.743 803.175 687.85 810.434 689.099C824.914 691.617 836.272 691.773 845.054 691.831C852.469 691.89 858.695 691.617 863.222 691.324C863.749 686.64 864.257 681.898 864.725 677.117C865.72 667.047 866.54 657.153 867.223 647.474C861.134 647.396 854.733 647.142 848.039 646.674C842.927 646.303 837.989 645.815 833.267 645.269C826.846 646.147 818.24 647.006 808.053 647.064C778.586 647.201 757.178 640.507 751.87 638.77C741.917 635.511 733.955 631.862 728.452 629.052C729.428 632.682 730.696 637.853 731.808 644.117C733.135 651.553 736.238 671.458 733.174 701.784C731.125 722.06 730.091 732.423 725.368 744.854C721.309 755.567 710.322 778.888 680.035 799.144L680.074 799.203Z" fill="#0050C8"/>
<path d="M493.94 571.483C489.179 567.034 474.133 552.846 461.233 537.625C458.15 533.995 453.017 527.867 446.89 519.183C444.86 516.295 442.733 513.153 440.547 509.718C435.395 501.62 428.682 490.925 422.789 475.626C419.139 466.141 416.934 457.847 415.568 451.7C411.88 469.225 412.465 483.783 413.578 493.306C414.846 504.274 417.246 512.255 419.315 518.968C421.54 526.208 424.857 535.497 429.931 545.918C436.937 549.49 442.967 552.28 447.553 554.31C460.453 560.028 470.756 563.677 477.45 566.019C484.144 568.38 489.823 570.195 493.96 571.483H493.94Z" fill="#003C8D"/>
<path d="M668.582 578.488C670.768 569.18 674.593 562.798 677.13 559.227C684.135 549.333 692.507 545.586 701.855 541.293C709.407 537.819 715.437 535.165 720.999 537.78C722.014 538.248 722.482 538.658 728.337 542.425C730.366 543.771 733.333 545.703 736.982 547.908C746.525 553.685 784.618 575.854 840.177 582.294C877.822 586.665 906.762 581.864 923.467 578.957C957.599 573.024 985.271 562.545 1005 553.392C994.15 582.001 979.26 598.999 970.186 607.8C935.371 641.58 892.731 644.976 865.839 647.591C859.79 648.176 838.518 649.757 809.344 645.269C745.822 635.511 697.503 602.824 668.582 578.508V578.488Z" fill="#8BC7FE"/>
<path d="M664.369 591.27C671.004 599.311 678.791 607.839 687.826 616.425C700.764 628.72 713.391 638.38 724.631 645.912C724.475 642.595 723.89 638.341 722.172 633.794C721.665 632.467 721.119 631.257 720.572 630.144C726.017 633.15 732.008 636.018 738.487 638.536C769.477 650.537 798.183 649.386 817.874 646.225C791.353 642.653 756.792 634.262 720.143 614.318C707.361 607.351 695.964 599.877 685.953 592.441C685.484 592.09 681.874 589.202 676.8 585.221C673.404 582.567 670.614 580.381 668.779 578.957C667.296 583.074 665.813 587.172 664.33 591.29L664.369 591.27Z" fill="#00398B"/>
<path d="M992.766 578.937C975.417 607.683 953.073 622.065 944.135 627.237C936.231 631.803 924.366 637.56 908.91 641.092C908.715 641.151 888.81 646.4 853.956 648.235C848.863 648.508 843.887 648.567 843.887 648.567C843.887 648.567 837.759 648.645 831.436 648.391C776.54 646.283 723.362 623.88 723.362 623.88C715.146 620.641 701.642 614.864 687.747 606.434C675.511 599.018 668.74 593.066 665.149 589.709C663.432 588.09 662.046 586.704 661.109 585.728C668.447 572.77 675.785 559.832 683.142 546.874C690.011 553.743 700.198 563.228 713.468 573.024C732.007 586.704 785.107 625.949 857.059 623.587C937.5 620.972 991.069 568.028 1005 553.333C1002.6 559.851 998.777 568.926 992.747 578.898L992.766 578.937Z" fill="#0087F7"/>
<path d="M649.906 574.117C651.233 571.268 653.146 567.423 655.761 563.111C656.971 561.12 661.323 553.997 666.533 548.416C670.67 543.986 682.047 531.789 698.733 531.282C708.119 530.989 715.847 534.502 721.019 537.761C717.604 538.073 712.94 538.736 707.534 540.239C701.796 541.859 695.279 543.323 689.502 547.909C681.462 554.29 677.013 562.896 668.582 580.791C666.884 584.421 665.577 587.426 664.757 589.358C661.596 586.294 658.415 583.094 655.195 579.737C653.38 577.845 651.624 575.971 649.906 574.098V574.117Z" fill="#0051CB"/>
<svg width="32" height="32" viewBox="0 0 32 32" fill="none" xmlns="http://www.w3.org/2000/svg">
<g clip-path="url(#clip0_286_152)">
<path d="M12.5378 28.7972C12.5378 30.2972 14.2889 30.2982 14.7498 29.4999C15.0384 28.9999 15.2498 28.4999 15.2498 26.9999C15.2498 25.4999 14.5258 24.2384 13.8296 22.8794C13.6504 22.5296 13.4578 22.1537 13.253 21.7309C12.9016 21.0056 12.4714 20.3367 11.8469 19.8035C11.7511 19.7217 11.7231 19.5023 11.7404 19.3555C11.8085 18.7759 11.8827 18.1969 11.9568 17.6179C12.0794 16.6609 12.202 15.7039 12.2975 14.7442C12.3645 14.0718 12.3633 13.378 12.2786 12.7082C12.0956 11.2586 11.2821 10.1946 10.0335 9.47786C9.6702 9.26926 9.56307 9.03637 9.60844 8.64632C9.73481 7.56183 10.1635 6.58265 10.7083 5.66605C11.2087 4.82425 11.7757 4.02144 12.3417 3.2201C12.4142 3.11747 12.4866 3.01486 12.5589 2.9122C12.6515 2.78077 12.8805 2.68113 13.049 2.67687C15.103 2.62543 17.1573 2.59121 19.2117 2.56589C19.3528 2.56407 19.5394 2.62989 19.6303 2.73115C20.9118 4.1585 22.0569 5.678 22.7442 7.49459C22.7912 7.61894 22.8303 7.74673 22.8661 7.87492C23.1241 8.79206 23.1199 8.79508 22.3848 9.31247L22.3515 9.33589C20.9673 10.3108 20.4612 11.705 20.5228 13.3373C20.5933 15.2138 21.0153 17.0328 21.4965 18.8385C21.5943 19.2054 21.6577 19.4768 21.3049 19.8152C20.5985 20.4924 20.3047 21.4218 20.1641 22.3672C20.0179 23.352 19.8995 24.3409 19.7811 25.3299C19.7154 25.8788 19.7122 26.3421 19.5792 26.9759C19.0702 29.4029 17.7498 31.9999 15.7498 31.9999H12.5378C11.2498 31.9999 10.2498 30.7972 10.2498 28.7972H12.5378Z" fill="#155DFC"/>
<path d="M11.5607 1.64008C10.8347 2.775 10.091 3.85078 9.44034 4.98022C8.85324 5.9991 8.46218 7.10566 8.27283 8.28027C8.02494 9.8182 8.4235 11.1797 9.51913 12.2444C10.4211 13.1209 10.6538 14.1118 10.538 15.2919C10.2857 17.8647 9.77936 20.385 9.05131 22.863C9.04422 22.8869 9.02214 22.9064 8.97435 22.9744C8.53792 22.8381 8.07901 22.7111 7.63165 22.5524C1.38253 20.3344 -3.18062 16.2382 -6.04606 10.2575C-6.29374 9.74063 -6.51975 9.21246 -6.73158 8.68004C-6.77189 8.57878 -6.74374 8.38234 -6.66961 8.31328C-4.28698 6.09307 -1.91002 3.86617 0.50117 1.67735C2.16142 0.170403 4.14529 -0.303895 6.31143 0.184986C7.98444 0.562481 9.62828 1.06959 11.2841 1.52262C11.3961 1.5532 11.5 1.61376 11.5607 1.64008Z" fill="#155DFC"/>
<path d="M24.2397 22.3314C23.8972 21.1864 23.5199 20.1003 23.2528 18.9878C22.9016 17.5244 22.6171 16.0438 22.3435 14.5632C22.1888 13.7264 22.35 12.9544 22.9725 12.3027C24.5667 10.6339 24.5983 8.69929 23.8328 6.67592C23.1868 4.96849 22.1482 3.49759 20.9256 2.15226C20.7825 1.9947 20.6423 1.8341 20.4795 1.65123C20.5718 1.6006 20.6427 1.54693 20.7227 1.52C22.532 0.910212 24.3624 0.387106 26.2602 0.122211C28.1953 -0.147949 29.8448 0.454544 31.2612 1.74398C32.5042 2.87545 33.7011 4.05816 34.9529 5.17971C35.9556 6.07788 37.0032 6.92664 38.0476 7.77681C38.2361 7.93012 38.2841 8.05386 38.2282 8.27602C37.2646 12.1148 35.328 15.3721 32.3643 17.9969C30.0283 20.0657 27.3376 21.4946 24.2397 22.3316V22.3314Z" fill="#155DFC"/>
</g>
<defs>
<clipPath id="clip0_221_160">
<rect width="1000" height="1000" fill="white"/>
<clipPath id="clip0_286_152">
<rect width="32" height="32" rx="6" fill="white"/>
</clipPath>
</defs>
</svg>

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 3.0 KiB

View File

@@ -1,4 +1,4 @@
import { App as AntdApp, ConfigProvider } from 'antd';
import { App as AntdApp, ConfigProvider, theme } from 'antd';
import { useEffect, useState } from 'react';
import { BrowserRouter, Route } from 'react-router';
import { Routes } from 'react-router';
@@ -7,10 +7,12 @@ import { userApi } from './entity/users';
import { AuthPageComponent } from './pages/AuthPageComponent';
import { OAuthCallbackPage } from './pages/OAuthCallbackPage';
import { OauthStorageComponent } from './pages/OauthStorageComponent';
import { ThemeProvider, useTheme } from './shared/theme';
import { MainScreenComponent } from './widgets/main/MainScreenComponent';
function App() {
function AppContent() {
const [isAuthorized, setIsAuthorized] = useState(false);
const { resolvedTheme } = useTheme();
useEffect(() => {
const isAuthorized = userApi.isAuthorized();
@@ -24,6 +26,7 @@ function App() {
return (
<ConfigProvider
theme={{
algorithm: resolvedTheme === 'dark' ? theme.darkAlgorithm : theme.defaultAlgorithm,
token: {
colorPrimary: '#155dfc', // Tailwind blue-600
},
@@ -45,4 +48,12 @@ function App() {
);
}
function App() {
return (
<ThemeProvider>
<AppContent />
</ThemeProvider>
);
}
export default App;

View File

@@ -0,0 +1,528 @@
import { describe, expect, it } from 'vitest';
import {
ConnectionStringParser,
type ParseError,
type ParseResult,
} from './ConnectionStringParser';
describe('ConnectionStringParser', () => {
// Helper to assert successful parse
const expectSuccess = (result: ParseResult | ParseError): ParseResult => {
expect('error' in result).toBe(false);
return result as ParseResult;
};
// Helper to assert parse error
const expectError = (result: ParseResult | ParseError): ParseError => {
expect('error' in result).toBe(true);
return result as ParseError;
};
describe('Standard PostgreSQL URI (postgresql://)', () => {
it('should parse basic postgresql:// connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://myuser:mypassword@localhost:5432/mydb'),
);
expect(result.host).toBe('localhost');
expect(result.port).toBe(5432);
expect(result.username).toBe('myuser');
expect(result.password).toBe('mypassword');
expect(result.database).toBe('mydb');
expect(result.isHttps).toBe(false);
});
it('should default port to 5432 when not specified', () => {
const result = expectSuccess(ConnectionStringParser.parse('postgresql://user:pass@host/db'));
expect(result.port).toBe(5432);
});
it('should handle URL-encoded passwords', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://user:p%40ss%23word@host:5432/db'),
);
expect(result.password).toBe('p@ss#word');
});
it('should handle URL-encoded usernames', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://user%40domain:password@host:5432/db'),
);
expect(result.username).toBe('user@domain');
});
});
describe('Postgres URI (postgres://)', () => {
it('should parse basic postgres:// connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgres://admin:secret@db.example.com:5432/production'),
);
expect(result.host).toBe('db.example.com');
expect(result.port).toBe(5432);
expect(result.username).toBe('admin');
expect(result.password).toBe('secret');
expect(result.database).toBe('production');
});
});
describe('Supabase Direct Connection', () => {
it('should parse Supabase direct connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgresql://postgres:mySecretPassword@db.abcdefghijklmnop.supabase.co:5432/postgres',
),
);
expect(result.host).toBe('db.abcdefghijklmnop.supabase.co');
expect(result.port).toBe(5432);
expect(result.username).toBe('postgres');
expect(result.password).toBe('mySecretPassword');
expect(result.database).toBe('postgres');
});
});
describe('Supabase Pooler Connection', () => {
it('should parse Supabase pooler session mode connection string (port 5432)', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgres://postgres.abcdefghijklmnop:myPassword@aws-0-us-east-1.pooler.supabase.com:5432/postgres',
),
);
expect(result.host).toBe('aws-0-us-east-1.pooler.supabase.com');
expect(result.port).toBe(5432);
expect(result.username).toBe('postgres.abcdefghijklmnop');
expect(result.password).toBe('myPassword');
expect(result.database).toBe('postgres');
});
it('should parse Supabase pooler transaction mode connection string (port 6543)', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgres://postgres.projectref:myPassword@aws-0-eu-west-1.pooler.supabase.com:6543/postgres',
),
);
expect(result.host).toBe('aws-0-eu-west-1.pooler.supabase.com');
expect(result.port).toBe(6543);
expect(result.username).toBe('postgres.projectref');
});
});
describe('JDBC Connection String', () => {
it('should parse JDBC connection string with user and password params', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'jdbc:postgresql://localhost:5432/mydb?user=admin&password=secret',
),
);
expect(result.host).toBe('localhost');
expect(result.port).toBe(5432);
expect(result.username).toBe('admin');
expect(result.password).toBe('secret');
expect(result.database).toBe('mydb');
});
it('should parse JDBC connection string without port', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'jdbc:postgresql://db.example.com/mydb?user=admin&password=secret',
),
);
expect(result.host).toBe('db.example.com');
expect(result.port).toBe(5432);
});
it('should parse JDBC with sslmode parameter', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'jdbc:postgresql://host:5432/db?user=u&password=p&sslmode=require',
),
);
expect(result.isHttps).toBe(true);
});
it('should return error for JDBC without user parameter', () => {
const result = expectError(
ConnectionStringParser.parse('jdbc:postgresql://host:5432/db?password=secret'),
);
expect(result.error).toContain('user');
expect(result.format).toBe('JDBC');
});
it('should return error for JDBC without password parameter', () => {
const result = expectError(
ConnectionStringParser.parse('jdbc:postgresql://host:5432/db?user=admin'),
);
expect(result.error).toContain('Password');
expect(result.format).toBe('JDBC');
});
});
describe('Neon Connection String', () => {
it('should parse Neon connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgresql://neonuser:password123@ep-cool-name-123456.us-east-2.aws.neon.tech/neondb',
),
);
expect(result.host).toBe('ep-cool-name-123456.us-east-2.aws.neon.tech');
expect(result.username).toBe('neonuser');
expect(result.database).toBe('neondb');
});
});
describe('Railway Connection String', () => {
it('should parse Railway connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgresql://postgres:railwaypass@containers-us-west-123.railway.app:5432/railway',
),
);
expect(result.host).toBe('containers-us-west-123.railway.app');
expect(result.username).toBe('postgres');
expect(result.database).toBe('railway');
});
});
describe('Render Connection String', () => {
it('should parse Render connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgresql://renderuser:renderpass@dpg-abc123.oregon-postgres.render.com/mydb',
),
);
expect(result.host).toBe('dpg-abc123.oregon-postgres.render.com');
expect(result.username).toBe('renderuser');
expect(result.database).toBe('mydb');
});
});
describe('DigitalOcean Connection String', () => {
it('should parse DigitalOcean connection string with sslmode', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgresql://doadmin:dopassword@db-postgresql-nyc1-12345-do-user-123456-0.b.db.ondigitalocean.com:25060/defaultdb?sslmode=require',
),
);
expect(result.host).toBe('db-postgresql-nyc1-12345-do-user-123456-0.b.db.ondigitalocean.com');
expect(result.port).toBe(25060);
expect(result.username).toBe('doadmin');
expect(result.database).toBe('defaultdb');
expect(result.isHttps).toBe(true);
});
});
describe('AWS RDS Connection String', () => {
it('should parse AWS RDS connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgresql://rdsuser:rdspass@mydb.abc123xyz.us-east-1.rds.amazonaws.com:5432/mydb',
),
);
expect(result.host).toBe('mydb.abc123xyz.us-east-1.rds.amazonaws.com');
expect(result.username).toBe('rdsuser');
});
});
describe('Azure Database for PostgreSQL Connection String', () => {
it('should parse Azure connection string with user@server format', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgresql://myuser@myserver:mypassword@myserver.postgres.database.azure.com:5432/mydb?sslmode=require',
),
);
expect(result.host).toBe('myserver.postgres.database.azure.com');
expect(result.port).toBe(5432);
expect(result.username).toBe('myuser');
expect(result.password).toBe('mypassword');
expect(result.database).toBe('mydb');
expect(result.isHttps).toBe(true);
});
});
describe('Heroku Connection String', () => {
it('should parse Heroku connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgres://herokuuser:herokupass@ec2-12-34-56-789.compute-1.amazonaws.com:5432/herokudb',
),
);
expect(result.host).toBe('ec2-12-34-56-789.compute-1.amazonaws.com');
expect(result.username).toBe('herokuuser');
expect(result.database).toBe('herokudb');
});
});
describe('CockroachDB Connection String', () => {
it('should parse CockroachDB connection string with sslmode=verify-full', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgresql://crdbuser:crdbpass@free-tier.gcp-us-central1.cockroachlabs.cloud:26257/defaultdb?sslmode=verify-full',
),
);
expect(result.host).toBe('free-tier.gcp-us-central1.cockroachlabs.cloud');
expect(result.port).toBe(26257);
expect(result.isHttps).toBe(true);
});
});
describe('SSL Mode Handling', () => {
it('should set isHttps=true for sslmode=require', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://u:p@host:5432/db?sslmode=require'),
);
expect(result.isHttps).toBe(true);
});
it('should set isHttps=true for sslmode=verify-ca', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://u:p@host:5432/db?sslmode=verify-ca'),
);
expect(result.isHttps).toBe(true);
});
it('should set isHttps=true for sslmode=verify-full', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://u:p@host:5432/db?sslmode=verify-full'),
);
expect(result.isHttps).toBe(true);
});
it('should set isHttps=false for sslmode=disable', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://u:p@host:5432/db?sslmode=disable'),
);
expect(result.isHttps).toBe(false);
});
it('should set isHttps=false when no sslmode specified', () => {
const result = expectSuccess(ConnectionStringParser.parse('postgresql://u:p@host:5432/db'));
expect(result.isHttps).toBe(false);
});
});
describe('libpq Key-Value Format', () => {
it('should parse libpq format connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'host=localhost port=5432 dbname=mydb user=admin password=secret',
),
);
expect(result.host).toBe('localhost');
expect(result.port).toBe(5432);
expect(result.username).toBe('admin');
expect(result.password).toBe('secret');
expect(result.database).toBe('mydb');
});
it('should parse libpq format with quoted password containing spaces', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
"host=localhost port=5432 dbname=mydb user=admin password='my secret pass'",
),
);
expect(result.password).toBe('my secret pass');
});
it('should default port to 5432 when not specified in libpq format', () => {
const result = expectSuccess(
ConnectionStringParser.parse('host=localhost dbname=mydb user=admin password=secret'),
);
expect(result.port).toBe(5432);
});
it('should handle hostaddr as alternative to host', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'hostaddr=192.168.1.1 port=5432 dbname=mydb user=admin password=secret',
),
);
expect(result.host).toBe('192.168.1.1');
});
it('should handle database as alternative to dbname', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'host=localhost port=5432 database=mydb user=admin password=secret',
),
);
expect(result.database).toBe('mydb');
});
it('should handle username as alternative to user', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'host=localhost port=5432 dbname=mydb username=admin password=secret',
),
);
expect(result.username).toBe('admin');
});
it('should parse sslmode in libpq format', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'host=localhost dbname=mydb user=admin password=secret sslmode=require',
),
);
expect(result.isHttps).toBe(true);
});
it('should return error for libpq format missing host', () => {
const result = expectError(
ConnectionStringParser.parse('port=5432 dbname=mydb user=admin password=secret'),
);
expect(result.error).toContain('Host');
expect(result.format).toBe('libpq');
});
it('should return error for libpq format missing user', () => {
const result = expectError(
ConnectionStringParser.parse('host=localhost dbname=mydb password=secret'),
);
expect(result.error).toContain('Username');
expect(result.format).toBe('libpq');
});
it('should return error for libpq format missing password', () => {
const result = expectError(
ConnectionStringParser.parse('host=localhost dbname=mydb user=admin'),
);
expect(result.error).toContain('Password');
expect(result.format).toBe('libpq');
});
it('should return error for libpq format missing dbname', () => {
const result = expectError(
ConnectionStringParser.parse('host=localhost user=admin password=secret'),
);
expect(result.error).toContain('Database');
expect(result.format).toBe('libpq');
});
});
describe('Error Cases', () => {
it('should return error for empty string', () => {
const result = expectError(ConnectionStringParser.parse(''));
expect(result.error).toContain('empty');
});
it('should return error for whitespace-only string', () => {
const result = expectError(ConnectionStringParser.parse(' '));
expect(result.error).toContain('empty');
});
it('should return error for unrecognized format', () => {
const result = expectError(ConnectionStringParser.parse('some random text'));
expect(result.error).toContain('Unrecognized');
});
it('should return error for missing username in URI', () => {
const result = expectError(
ConnectionStringParser.parse('postgresql://:password@host:5432/db'),
);
expect(result.error).toContain('Username');
});
it('should return error for missing password in URI', () => {
const result = expectError(ConnectionStringParser.parse('postgresql://user@host:5432/db'));
expect(result.error).toContain('Password');
});
it('should return error for missing database in URI', () => {
const result = expectError(ConnectionStringParser.parse('postgresql://user:pass@host:5432/'));
expect(result.error).toContain('Database');
});
it('should return error for invalid JDBC format', () => {
const result = expectError(ConnectionStringParser.parse('jdbc:postgresql://invalid'));
expect(result.format).toBe('JDBC');
});
});
describe('Edge Cases', () => {
it('should handle special characters in password', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://user:p%40ss%3Aw%2Ford@host:5432/db'),
);
expect(result.password).toBe('p@ss:w/ord');
});
it('should handle numeric database names', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://user:pass@host:5432/12345'),
);
expect(result.database).toBe('12345');
});
it('should handle hyphenated host names', () => {
const result = expectSuccess(
ConnectionStringParser.parse('postgresql://user:pass@my-database-host.example.com:5432/db'),
);
expect(result.host).toBe('my-database-host.example.com');
});
it('should handle connection string with extra query parameters', () => {
const result = expectSuccess(
ConnectionStringParser.parse(
'postgresql://user:pass@host:5432/db?sslmode=require&connect_timeout=10&application_name=myapp',
),
);
expect(result.isHttps).toBe(true);
expect(result.database).toBe('db');
});
it('should trim whitespace from connection string', () => {
const result = expectSuccess(
ConnectionStringParser.parse(' postgresql://user:pass@host:5432/db '),
);
expect(result.host).toBe('host');
});
});
});

View File

@@ -0,0 +1,284 @@
export type ParseResult = {
host: string;
port: number;
username: string;
password: string;
database: string;
isHttps: boolean;
};
export type ParseError = {
error: string;
format?: string;
};
export class ConnectionStringParser {
/**
* Parses a PostgreSQL connection string in various formats.
*
* Supported formats:
* 1. Standard PostgreSQL URI: postgresql://user:pass@host:port/db
* 2. Postgres URI: postgres://user:pass@host:port/db
* 3. Supabase Direct: postgresql://postgres:pass@db.xxx.supabase.co:5432/postgres
* 4. Supabase Pooler Session: postgres://postgres.ref:pass@aws-0-region.pooler.supabase.com:5432/postgres
* 5. Supabase Pooler Transaction: same as above with port 6543
* 6. JDBC: jdbc:postgresql://host:port/db?user=x&password=y
* 7. Neon: postgresql://user:pass@ep-xxx.neon.tech/db
* 8. Railway: postgresql://postgres:pass@xxx.railway.app:port/railway
* 9. Render: postgresql://user:pass@xxx.render.com/db
* 10. DigitalOcean: postgresql://user:pass@xxx.ondigitalocean.com:port/db?sslmode=require
* 11. AWS RDS: postgresql://user:pass@xxx.rds.amazonaws.com:port/db
* 12. Azure: postgresql://user@server:pass@xxx.postgres.database.azure.com:port/db?sslmode=require
* 13. Heroku: postgres://user:pass@ec2-xxx.amazonaws.com:port/db
* 14. CockroachDB: postgresql://user:pass@xxx.cockroachlabs.cloud:port/db?sslmode=verify-full
* 15. With SSL params: postgresql://user:pass@host:port/db?sslmode=require
* 16. libpq key-value: host=x port=5432 dbname=db user=u password=p
*/
static parse(connectionString: string): ParseResult | ParseError {
const trimmed = connectionString.trim();
if (!trimmed) {
return { error: 'Connection string is empty' };
}
// Try JDBC format first (starts with jdbc:)
if (trimmed.startsWith('jdbc:postgresql://')) {
return this.parseJdbc(trimmed);
}
// Try libpq key-value format (contains key=value pairs without ://)
if (this.isLibpqFormat(trimmed)) {
return this.parseLibpq(trimmed);
}
// Try URI format (postgresql:// or postgres://)
if (trimmed.startsWith('postgresql://') || trimmed.startsWith('postgres://')) {
return this.parseUri(trimmed);
}
return {
error: 'Unrecognized connection string format',
};
}
private static isLibpqFormat(str: string): boolean {
// libpq format has key=value pairs separated by spaces
// Must contain at least host= or dbname= to be considered libpq format
return (
!str.includes('://') &&
(str.includes('host=') || str.includes('dbname=')) &&
str.includes('=')
);
}
private static parseUri(connectionString: string): ParseResult | ParseError {
try {
// Handle Azure format where username contains @: user@server:pass
// Azure format: postgresql://user@servername:password@host:port/db
const azureMatch = connectionString.match(
/^postgres(?:ql)?:\/\/([^@:]+)@([^:]+):([^@]+)@([^:/?]+):?(\d+)?\/([^?]+)(?:\?(.*))?$/,
);
if (azureMatch) {
const [, user, , password, host, port, database, queryString] = azureMatch;
const isHttps = this.checkSslMode(queryString);
return {
host: host,
port: port ? parseInt(port, 10) : 5432,
username: decodeURIComponent(user),
password: decodeURIComponent(password),
database: decodeURIComponent(database),
isHttps,
};
}
// Standard URI parsing using URL API
const url = new URL(connectionString);
const host = url.hostname;
const port = url.port ? parseInt(url.port, 10) : 5432;
const username = decodeURIComponent(url.username);
const password = decodeURIComponent(url.password);
const database = decodeURIComponent(url.pathname.slice(1)); // Remove leading /
const isHttps = this.checkSslMode(url.search);
// Validate required fields
if (!host) {
return { error: 'Host is missing from connection string' };
}
if (!username) {
return { error: 'Username is missing from connection string' };
}
if (!password) {
return { error: 'Password is missing from connection string' };
}
if (!database) {
return { error: 'Database name is missing from connection string' };
}
return {
host,
port,
username,
password,
database,
isHttps,
};
} catch (e) {
return {
error: `Failed to parse connection string: ${(e as Error).message}`,
format: 'URI',
};
}
}
private static parseJdbc(connectionString: string): ParseResult | ParseError {
try {
// JDBC format: jdbc:postgresql://host:port/database?user=x&password=y
const jdbcRegex = /^jdbc:postgresql:\/\/([^:/?]+):?(\d+)?\/([^?]+)(?:\?(.*))?$/;
const match = connectionString.match(jdbcRegex);
if (!match) {
return {
error:
'Invalid JDBC connection string format. Expected: jdbc:postgresql://host:port/database?user=x&password=y',
format: 'JDBC',
};
}
const [, host, port, database, queryString] = match;
if (!queryString) {
return {
error: 'JDBC connection string is missing query parameters (user and password)',
format: 'JDBC',
};
}
const params = new URLSearchParams(queryString);
const username = params.get('user');
const password = params.get('password');
const isHttps = this.checkSslMode(queryString);
if (!username) {
return {
error: 'Username (user parameter) is missing from JDBC connection string',
format: 'JDBC',
};
}
if (!password) {
return {
error: 'Password parameter is missing from JDBC connection string',
format: 'JDBC',
};
}
return {
host,
port: port ? parseInt(port, 10) : 5432,
username: decodeURIComponent(username),
password: decodeURIComponent(password),
database: decodeURIComponent(database),
isHttps,
};
} catch (e) {
return {
error: `Failed to parse JDBC connection string: ${(e as Error).message}`,
format: 'JDBC',
};
}
}
private static parseLibpq(connectionString: string): ParseResult | ParseError {
try {
// libpq format: host=x port=5432 dbname=db user=u password=p
// Values can be quoted with single quotes: password='my pass'
const params: Record<string, string> = {};
// Match key=value or key='quoted value'
const regex = /(\w+)=(?:'([^']*)'|(\S+))/g;
let match;
while ((match = regex.exec(connectionString)) !== null) {
const key = match[1];
const value = match[2] !== undefined ? match[2] : match[3];
params[key] = value;
}
const host = params['host'] || params['hostaddr'];
const port = params['port'];
const database = params['dbname'] || params['database'];
const username = params['user'] || params['username'];
const password = params['password'];
const sslmode = params['sslmode'];
if (!host) {
return {
error: 'Host is missing from connection string. Use host=hostname',
format: 'libpq',
};
}
if (!username) {
return {
error: 'Username is missing from connection string. Use user=username',
format: 'libpq',
};
}
if (!password) {
return {
error: 'Password is missing from connection string. Use password=yourpassword',
format: 'libpq',
};
}
if (!database) {
return {
error: 'Database name is missing from connection string. Use dbname=database',
format: 'libpq',
};
}
const isHttps = this.isSslEnabled(sslmode);
return {
host,
port: port ? parseInt(port, 10) : 5432,
username,
password,
database,
isHttps,
};
} catch (e) {
return {
error: `Failed to parse libpq connection string: ${(e as Error).message}`,
format: 'libpq',
};
}
}
private static checkSslMode(queryString: string | undefined | null): boolean {
if (!queryString) return false;
const params = new URLSearchParams(
queryString.startsWith('?') ? queryString.slice(1) : queryString,
);
const sslmode = params.get('sslmode');
return this.isSslEnabled(sslmode);
}
private static isSslEnabled(sslmode: string | null | undefined): boolean {
if (!sslmode) return false;
// These modes require SSL
const sslModes = ['require', 'verify-ca', 'verify-full'];
return sslModes.includes(sslmode.toLowerCase());
}
}

View File

@@ -11,4 +11,10 @@ export interface PostgresqlDatabase {
password: string;
database?: string;
isHttps: boolean;
// backup settings
includeSchemas?: string[];
// restore settings (not saved to DB)
isExcludeExtensions?: boolean;
}

View File

@@ -9,6 +9,7 @@ export type { TelegramNotifier } from './models/telegram/TelegramNotifier';
export { validateTelegramNotifier } from './models/telegram/validateTelegramNotifier';
export type { WebhookNotifier } from './models/webhook/WebhookNotifier';
export type { WebhookHeader } from './models/webhook/WebhookHeader';
export { validateWebhookNotifier } from './models/webhook/validateWebhookNotifier';
export { WebhookMethod } from './models/webhook/WebhookMethod';

View File

@@ -1,6 +1,6 @@
import type { EmailNotifier } from './EmailNotifier';
export const validateEmailNotifier = (isCreate: boolean, notifier: EmailNotifier): boolean => {
export const validateEmailNotifier = (notifier: EmailNotifier): boolean => {
if (!notifier.targetEmail) {
return false;
}
@@ -13,9 +13,5 @@ export const validateEmailNotifier = (isCreate: boolean, notifier: EmailNotifier
return false;
}
if (isCreate && !notifier.smtpPassword) {
return false;
}
return true;
};

View File

@@ -0,0 +1,4 @@
export interface WebhookHeader {
key: string;
value: string;
}

View File

@@ -1,6 +1,9 @@
import type { WebhookHeader } from './WebhookHeader';
import type { WebhookMethod } from './WebhookMethod';
export interface WebhookNotifier {
webhookUrl: string;
webhookMethod: WebhookMethod;
bodyTemplate?: string;
headers?: WebhookHeader[];
}

View File

@@ -8,3 +8,6 @@ export { getStorageLogoFromType } from './models/getStorageLogoFromType';
export { getStorageNameFromType } from './models/getStorageNameFromType';
export { type GoogleDriveStorage } from './models/GoogleDriveStorage';
export { type AzureBlobStorage } from './models/AzureBlobStorage';
export { type FTPStorage } from './models/FTPStorage';
export { type SFTPStorage } from './models/SFTPStorage';
export { type RcloneStorage } from './models/RcloneStorage';

View File

@@ -0,0 +1,9 @@
export interface FTPStorage {
host: string;
port: number;
username: string;
password: string;
useSsl: boolean;
skipTlsVerify?: boolean;
path?: string;
}

View File

@@ -0,0 +1,4 @@
export interface RcloneStorage {
configContent: string;
remotePath?: string;
}

View File

@@ -6,4 +6,5 @@ export interface S3Storage {
s3Endpoint?: string;
s3Prefix?: string;
s3UseVirtualHostedStyle?: boolean;
skipTLSVerify?: boolean;
}

View File

@@ -0,0 +1,9 @@
export interface SFTPStorage {
host: string;
port: number;
username: string;
password?: string;
privateKey?: string;
path?: string;
skipHostKeyVerify?: boolean;
}

View File

@@ -1,8 +1,11 @@
import type { AzureBlobStorage } from './AzureBlobStorage';
import type { FTPStorage } from './FTPStorage';
import type { GoogleDriveStorage } from './GoogleDriveStorage';
import type { LocalStorage } from './LocalStorage';
import type { NASStorage } from './NASStorage';
import type { RcloneStorage } from './RcloneStorage';
import type { S3Storage } from './S3Storage';
import type { SFTPStorage } from './SFTPStorage';
import type { StorageType } from './StorageType';
export interface Storage {
@@ -18,4 +21,7 @@ export interface Storage {
googleDriveStorage?: GoogleDriveStorage;
nasStorage?: NASStorage;
azureBlobStorage?: AzureBlobStorage;
ftpStorage?: FTPStorage;
sftpStorage?: SFTPStorage;
rcloneStorage?: RcloneStorage;
}

View File

@@ -4,4 +4,7 @@ export enum StorageType {
GOOGLE_DRIVE = 'GOOGLE_DRIVE',
NAS = 'NAS',
AZURE_BLOB = 'AZURE_BLOB',
FTP = 'FTP',
SFTP = 'SFTP',
RCLONE = 'RCLONE',
}

View File

@@ -12,6 +12,12 @@ export const getStorageLogoFromType = (type: StorageType) => {
return '/icons/storages/nas.svg';
case StorageType.AZURE_BLOB:
return '/icons/storages/azure.svg';
case StorageType.FTP:
return '/icons/storages/ftp.svg';
case StorageType.SFTP:
return '/icons/storages/sftp.svg';
case StorageType.RCLONE:
return '/icons/storages/rclone.svg';
default:
return '';
}

View File

@@ -12,6 +12,12 @@ export const getStorageNameFromType = (type: StorageType) => {
return 'NAS';
case StorageType.AZURE_BLOB:
return 'Azure Blob Storage';
case StorageType.FTP:
return 'FTP';
case StorageType.SFTP:
return 'SFTP';
case StorageType.RCLONE:
return 'Rclone';
default:
return '';
}

View File

@@ -446,7 +446,9 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
render: (createdAt: string) => (
<div>
{dayjs.utc(createdAt).local().format(getUserTimeFormat().format)} <br />
<span className="text-gray-500">({dayjs.utc(createdAt).local().fromNow()})</span>
<span className="text-gray-500 dark:text-gray-400">
({dayjs.utc(createdAt).local().fromNow()})
</span>
</div>
),
sorter: (a, b) => dayjs(a.createdAt).unix() - dayjs(b.createdAt).unix(),
@@ -522,8 +524,8 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
}
return (
<div className="mt-5 w-full rounded-md bg-white p-3 shadow md:p-5">
<h2 className="text-lg font-bold md:text-xl">Backups</h2>
<div className="mt-5 w-full rounded-md bg-white p-3 shadow md:p-5 dark:bg-gray-800">
<h2 className="text-lg font-bold md:text-xl dark:text-white">Backups</h2>
{!isBackupConfigLoading && !backupConfig?.isBackupsEnabled && (
<div className="text-sm text-red-600 md:text-base">
@@ -558,16 +560,16 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
{backups.map((backup) => (
<div
key={backup.id}
className="mb-2 rounded-lg border border-gray-200 bg-white p-4 shadow-sm"
className="mb-2 rounded-lg border border-gray-200 bg-white p-4 shadow-sm dark:border-gray-700 dark:bg-gray-800"
>
<div className="space-y-3">
<div className="flex items-start justify-between">
<div>
<div className="text-xs text-gray-500">Created at</div>
<div className="text-xs text-gray-500 dark:text-gray-400">Created at</div>
<div className="text-sm font-medium">
{dayjs.utc(backup.createdAt).local().format(getUserTimeFormat().format)}
</div>
<div className="text-xs text-gray-500">
<div className="text-xs text-gray-500 dark:text-gray-400">
({dayjs.utc(backup.createdAt).local().fromNow()})
</div>
</div>
@@ -576,11 +578,11 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
<div className="grid grid-cols-2 gap-4">
<div>
<div className="text-xs text-gray-500">Size</div>
<div className="text-xs text-gray-500 dark:text-gray-400">Size</div>
<div className="text-sm font-medium">{formatSize(backup.backupSizeMb)}</div>
</div>
<div>
<div className="text-xs text-gray-500">Duration</div>
<div className="text-xs text-gray-500 dark:text-gray-400">Duration</div>
<div className="text-sm font-medium">
{formatDuration(backup.backupDurationMs)}
</div>
@@ -602,12 +604,12 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
</div>
)}
{!hasMore && backups.length > 0 && (
<div className="mt-3 text-center text-sm text-gray-500">
<div className="mt-3 text-center text-sm text-gray-500 dark:text-gray-400">
All backups loaded ({totalBackups} total)
</div>
)}
{!isBackupsLoading && backups.length === 0 && (
<div className="py-8 text-center text-gray-500">No backups yet</div>
<div className="py-8 text-center text-gray-500 dark:text-gray-400">No backups yet</div>
)}
</div>
@@ -628,7 +630,7 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
</div>
)}
{!hasMore && backups.length > 0 && (
<div className="mt-2 text-center text-gray-500">
<div className="mt-2 text-center text-gray-500 dark:text-gray-400">
All backups loaded ({totalBackups} total)
</div>
)}

View File

@@ -72,6 +72,7 @@ export const EditBackupConfigComponent = ({
const [storages, setStorages] = useState<Storage[]>([]);
const [isStoragesLoading, setIsStoragesLoading] = useState(false);
const [isShowCreateStorage, setShowCreateStorage] = useState(false);
const [storageSelectKey, setStorageSelectKey] = useState(0);
const [isShowWarn, setIsShowWarn] = useState(false);
@@ -397,6 +398,7 @@ export const EditBackupConfigComponent = ({
<div className="mb-1 min-w-[150px] sm:mb-0">Storage</div>
<div className="flex w-full items-center">
<Select
key={storageSelectKey}
value={backupConfig.storage?.id}
onChange={(storageId) => {
if (storageId.includes('create-new-storage')) {
@@ -527,9 +529,12 @@ export const EditBackupConfigComponent = ({
title="Add storage"
footer={<div />}
open={isShowCreateStorage}
onCancel={() => setShowCreateStorage(false)}
onCancel={() => {
setShowCreateStorage(false);
setStorageSelectKey((prev) => prev + 1);
}}
>
<div className="my-3 max-w-[275px] text-gray-500">
<div className="my-3 max-w-[275px] text-gray-500 dark:text-gray-400">
Storage - is a place where backups will be stored (local disk, S3, Google Drive, etc.)
</div>

View File

@@ -29,7 +29,7 @@ export const DatabaseCardComponent = ({
return (
<div
className={`mb-3 cursor-pointer rounded p-3 shadow ${selectedDatabaseId === database.id ? 'bg-blue-100' : 'bg-white'}`}
className={`mb-3 cursor-pointer rounded p-3 shadow ${selectedDatabaseId === database.id ? 'bg-blue-100 dark:bg-blue-800' : 'bg-white dark:bg-gray-800'}`}
onClick={() => setSelectedDatabaseId(database.id)}
>
<div className="flex">
@@ -49,7 +49,7 @@ export const DatabaseCardComponent = ({
</div>
{storage && (
<div className="text-sm text-gray-500">
<div className="text-sm text-gray-500 dark:text-gray-400">
<span>Storage: </span>
<span className="inline-flex items-center">
{storage.name}{' '}
@@ -65,11 +65,13 @@ export const DatabaseCardComponent = ({
)}
{database.lastBackupTime && (
<div className="text-gray-500">Last backup {dayjs(database.lastBackupTime).fromNow()}</div>
<div className="text-gray-500 dark:text-gray-400">
Last backup {dayjs(database.lastBackupTime).fromNow()}
</div>
)}
{database.lastBackupErrorMessage && (
<div className="mt-1 flex items-center text-sm text-red-600 underline">
<div className="mt-1 flex items-center text-sm text-red-600 underline dark:text-red-400">
<InfoCircleOutlined className="mr-1" style={{ color: 'red' }} />
Has backup error
</div>

View File

@@ -51,14 +51,14 @@ export const DatabaseComponent = ({
>
<div className="flex">
<div
className={`mr-2 cursor-pointer rounded-tl-md rounded-tr-md px-6 py-2 ${currentTab === 'config' ? 'bg-white' : 'bg-gray-200'}`}
className={`mr-2 cursor-pointer rounded-tl-md rounded-tr-md px-6 py-2 ${currentTab === 'config' ? 'bg-white dark:bg-gray-800' : 'bg-gray-200 dark:bg-gray-700'}`}
onClick={() => setCurrentTab('config')}
>
Config
</div>
<div
className={`mr-2 cursor-pointer rounded-tl-md rounded-tr-md px-6 py-2 ${currentTab === 'backups' ? 'bg-white' : 'bg-gray-200'}`}
className={`mr-2 cursor-pointer rounded-tl-md rounded-tr-md px-6 py-2 ${currentTab === 'backups' ? 'bg-white dark:bg-gray-800' : 'bg-gray-200 dark:bg-gray-700'}`}
onClick={() => setCurrentTab('backups')}
>
Backups

View File

@@ -147,7 +147,7 @@ export const DatabaseConfigComponent = ({
};
return (
<div className="w-full rounded-tr-md rounded-br-md rounded-bl-md bg-white p-3 shadow sm:p-5">
<div className="w-full rounded-tr-md rounded-br-md rounded-bl-md bg-white p-3 shadow sm:p-5 dark:bg-gray-800">
{!isEditName ? (
<div className="mb-5 flex items-center text-xl font-bold sm:text-2xl">
{database.name}
@@ -184,7 +184,7 @@ export const DatabaseConfigComponent = ({
setEditDatabase(undefined);
}}
>
<CloseOutlined className="text-gray-500" />
<CloseOutlined className="text-gray-500 dark:text-gray-400" />
</Button>
</div>
</div>
@@ -216,7 +216,7 @@ export const DatabaseConfigComponent = ({
{database.lastBackupErrorMessage}
</div>
<div className="mt-3 text-sm text-gray-500">
<div className="mt-3 text-sm text-gray-500 dark:text-gray-400">
To clean this error (choose any):
<ul>
<li>- test connection via button below (even if you updated settings);</li>
@@ -370,7 +370,6 @@ export const DatabaseConfigComponent = ({
<Button
type="primary"
className="w-full sm:mr-1 sm:w-auto"
ghost
onClick={testConnection}
loading={isTestingConnection}
disabled={isTestingConnection}
@@ -381,7 +380,6 @@ export const DatabaseConfigComponent = ({
<Button
type="primary"
className="w-full sm:mr-1 sm:w-auto"
ghost
onClick={copyDatabase}
loading={isCopying}
disabled={isCopying}

View File

@@ -47,7 +47,7 @@ export const DatabasesComponent = ({ contentHeight, workspace, isCanManageDBs }:
if (selectDatabaseId) {
updateSelectedDatabaseId(selectDatabaseId);
} else if (!selectedDatabaseId && !isSilent && !isMobile) {
// On desktop, auto-select a database; on mobile, keep it unselected
// On desktop, auto-select a database; on mobile, keep it unselected to show the list first
const savedDatabaseId = localStorage.getItem(
`${SELECTED_DATABASE_STORAGE_KEY}_${workspace.id}`,
);
@@ -111,7 +111,7 @@ export const DatabasesComponent = ({ contentHeight, workspace, isCanManageDBs }:
placeholder="Search database"
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="w-full border-b border-gray-300 p-1 text-gray-500 outline-none"
className="w-full border-b border-gray-300 p-1 text-gray-500 outline-none dark:text-gray-400"
/>
</div>
</>
@@ -127,14 +127,14 @@ export const DatabasesComponent = ({ contentHeight, workspace, isCanManageDBs }:
/>
))
: searchQuery && (
<div className="mb-4 text-center text-sm text-gray-500">
<div className="mb-4 text-center text-sm text-gray-500 dark:text-gray-400">
No databases found matching &quot;{searchQuery}&quot;
</div>
)}
{databases.length < 5 && isCanManageDBs && addDatabaseButton}
<div className="mx-3 text-center text-xs text-gray-500">
<div className="mx-3 text-center text-xs text-gray-500 dark:text-gray-400">
Database - is a thing we are backing up
</div>
</div>

View File

@@ -98,7 +98,12 @@ export const CreateReadOnlyComponent = ({
<p className="mb-2">
Postgresus enforce enterprise-grade security (
<a href="https://postgresus.com/security" target="_blank" rel="noreferrer">
<a
href="https://postgresus.com/security"
target="_blank"
rel="noreferrer"
className="!text-blue-600 dark:!text-blue-400"
>
read in details here
</a>
). However, it is not possible to be covered from all possible risks.

View File

@@ -43,6 +43,7 @@ export const EditDatabaseNotifiersComponent = ({
const [notifiers, setNotifiers] = useState<Notifier[]>([]);
const [isNotifiersLoading, setIsNotifiersLoading] = useState(false);
const [isShowCreateNotifier, setShowCreateNotifier] = useState(false);
const [notifierSelectKey, setNotifierSelectKey] = useState(0);
const saveDatabase = async () => {
if (!editingDatabase) return;
@@ -93,7 +94,7 @@ export const EditDatabaseNotifiersComponent = ({
return (
<div>
<div className="mb-5 max-w-[275px] text-gray-500">
<div className="mb-5 max-w-[275px] text-gray-500 dark:text-gray-400">
Notifier - is a place where notifications will be sent (email, Slack, Telegram, etc.)
<br />
<br />
@@ -104,6 +105,7 @@ export const EditDatabaseNotifiersComponent = ({
<div className="min-w-[150px]">Notifiers</div>
<Select
key={notifierSelectKey}
mode="multiple"
value={editingDatabase.notifiers.map((n) => n.id)}
onChange={(notifiersIds) => {
@@ -160,9 +162,12 @@ export const EditDatabaseNotifiersComponent = ({
title="Add notifier"
footer={<div />}
open={isShowCreateNotifier}
onCancel={() => setShowCreateNotifier(false)}
onCancel={() => {
setShowCreateNotifier(false);
setNotifierSelectKey((prev) => prev + 1);
}}
>
<div className="my-3 max-w-[275px] text-gray-500">
<div className="my-3 max-w-[275px] text-gray-500 dark:text-gray-400">
Notifier - is a place where notifications will be sent (email, Slack, Telegram, etc.)
</div>

Some files were not shown because too many files have changed in this diff Show More