Compare commits

...

25 Commits

Author SHA1 Message Date
Rostislav Dugin
30c2e2d156 Merge pull request #403 from databasus/develop
FIX (smtp): Add SMTP from field to env variables
2026-02-25 22:23:08 +03:00
Rostislav Dugin
ef7c5b45e6 FIX (smtp): Add SMTP from field to env variables 2026-02-25 22:13:04 +03:00
Rostislav Dugin
920c98e229 Merge pull request #397 from databasus/develop
FIX (migrations): Fix version of migrations tool goose
2026-02-22 23:43:55 +03:00
Rostislav Dugin
2a19a96aae FIX (migrations): Fix version of migrations tool goose 2026-02-22 23:43:23 +03:00
Rostislav Dugin
75aa2108d9 Merge pull request #396 from databasus/develop
FIX (email): Use current OS hostname instead of default localhost
2026-02-22 23:33:28 +03:00
Rostislav Dugin
0a0040839e FIX (email): Use current OS hostname instead of default localhost 2026-02-22 23:31:25 +03:00
Rostislav Dugin
ff4f795ece Merge pull request #394 from databasus/develop
FIX (nas): Add NAS share validation
2026-02-22 16:05:38 +03:00
Rostislav Dugin
dc05502580 FIX (nas): Add NAS share validation 2026-02-22 15:56:30 +03:00
Rostislav Dugin
1ca38f5583 Merge pull request #390 from databasus/develop
FEATURE (templates): Add PR template
2026-02-21 15:58:21 +03:00
Rostislav Dugin
40b3ff61c7 FEATURE (templates): Add PR template 2026-02-21 15:53:01 +03:00
Rostislav Dugin
e1b245a573 Merge pull request #389 from databasus/develop
Develop
2026-02-21 14:57:56 +03:00
Rostislav Dugin
fdf29b71f2 FIX (mongodb): Fix direct connection string parsing 2026-02-21 14:56:48 +03:00
Rostislav Dugin
49da981c21 Merge pull request #388 from databasus/main
Merge main into dev
2026-02-21 14:53:31 +03:00
Rostislav Dugin
9d611d3559 REFACTOR (mongodb): Refactor direct connection PR 2026-02-21 14:43:47 +03:00
ujstor
22cab53dab feature/mongodb-directConnection (#377)
FEATURE (mongodb): Add direct connection
2026-02-21 14:10:28 +03:00
Rostislav Dugin
d761c4156c Merge pull request #385 from databasus/develop
FIX (readme): Fix README typo
2026-02-20 17:17:45 +03:00
Rostislav Dugin
cbb8b82711 FIX (readme): Fix README typo 2026-02-20 17:01:44 +03:00
Rostislav Dugin
8e3d1e5bff Merge pull request #384 from databasus/develop
FIX (backups): Do not reload backups if request already in progress
2026-02-20 15:04:19 +03:00
Rostislav Dugin
349e7f0ee8 FIX (backups): Do not reload backups if request already in progress 2026-02-20 14:43:07 +03:00
Rostislav Dugin
3a274e135b Merge pull request #383 from databasus/develop
FEATURE (backups): Add GFS retention policy
2026-02-20 14:33:29 +03:00
Rostislav Dugin
61e937bc2a FEATURE (backups): Add GFS retention policy 2026-02-20 14:31:56 +03:00
Rostislav Dugin
f67919fe1a Merge pull request #374 from databasus/develop
FIX (backups): Fix backup download and clean up
2026-02-18 12:53:10 +03:00
Rostislav Dugin
91ee5966d8 FIX (backups): Fix backup download and clean up 2026-02-18 12:52:35 +03:00
Rostislav Dugin
d77d7d69a3 Merge pull request #371 from databasus/develop
FEATURE (backups): Add metadata alongsize with backup files itself to…
2026-02-17 19:54:53 +03:00
Rostislav Dugin
fc88b730d5 FEATURE (backups): Add metadata alongsize with backup files itself to make them recovarable without Databasus 2026-02-17 19:52:08 +03:00
44 changed files with 2513 additions and 450 deletions

38
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,38 @@
---
name: Bug Report
about: Report a bug or unexpected behavior in Databasus
labels: bug
---
## Databasus version
<!-- e.g. 1.4.2 -->
## Operating system and architecture
<!-- e.g. Ubuntu 22.04 x64, macOS 14 ARM, Windows 11 x64 -->
## Describe the bug (please write manually, do not ask AI to summarize)
**What happened:**
**What I expected:**
## Steps to reproduce
1.
2.
3.
## Have you asked AI how to solve the issue?
<!-- Using AI to diagnose issues before filing a bug report helps narrow down root causes. -->
- [ ] Claude Sonnet 4.6 or newer
- [ ] ChatGPT 5.2 or newer
- [ ] No
## Additional context / logs
<!-- Screenshots, error messages, relevant log output, etc. -->

View File

@@ -407,7 +407,7 @@ jobs:
- name: Run database migrations
run: |
cd backend
go install github.com/pressly/goose/v3/cmd/goose@latest
go install github.com/pressly/goose/v3/cmd/goose@v3.24.3
goose up
- name: Run Go tests

View File

@@ -50,6 +50,13 @@
- **Precise timing**: run backups at specific times (e.g., 4 AM during low traffic)
- **Smart compression**: 4-8x space savings with balanced compression (~20% overhead)
### 🗑️ **Retention policies**
- **Time period**: Keep backups for a fixed duration (e.g., 7 days, 3 months, 1 year)
- **Count**: Keep a fixed number of the most recent backups (e.g., last 30)
- **GFS (Grandfather-Father-Son)**: Layered retention — keep hourly, daily, weekly, monthly and yearly backups independently for fine-grained long-term history (enterprises requirement)
- **Size limits**: Set per-backup and total storage size caps to control storage usage
### 🗄️ **Multiple storage destinations** <a href="https://databasus.com/storages">(view supported)</a>
- **Local storage**: Keep backups on your VPS/server
@@ -69,6 +76,8 @@
- **Encryption for secrets**: Any sensitive data is encrypted and never exposed, even in logs or error messages
- **Read-only user**: Databasus uses a read-only user by default for backups and never stores anything that can modify your data
It is also important for Databasus that you are able to decrypt and restore backups from storages (local, S3, etc.) without Databasus itself. To do so, read our guide on [how to recover directly from storage](https://databasus.com/how-to-recover-without-databasus). We avoid "vendor lock-in" even to open source tool!
### 👥 **Suitable for teams** <a href="https://databasus.com/access-management">(docs)</a>
- **Workspaces**: Group databases, notifiers and storages for different projects or teams
@@ -218,8 +227,9 @@ For more options (NodePort, TLS, HTTPRoute for Gateway API), see the [Helm chart
3. **Configure schedule**: Choose from hourly, daily, weekly, monthly or cron intervals
4. **Set database connection**: Enter your database credentials and connection details
5. **Choose storage**: Select where to store your backups (local, S3, Google Drive, etc.)
6. **Add notifications** (optional): Configure email, Telegram, Slack, or webhook notifications
7. **Save and start**: Databasus will validate settings and begin the backup schedule
6. **Configure retention policy**: Choose time period, count or GFS to control how long backups are kept
7. **Add notifications** (optional): Configure email, Telegram, Slack, or webhook notifications
8. **Save and start**: Databasus will validate settings and begin the backup schedule
### 🔑 Resetting password <a href="https://databasus.com/password">(docs)</a>
@@ -231,56 +241,22 @@ docker exec -it databasus ./main --new-password="YourNewSecurePassword123" --ema
Replace `admin` with the actual email address of the user whose password you want to reset.
### 💾 Backuping Databasus itself
After installation, it is also recommended to <a href="https://databasus.com/faq/#backup-databasus">backup your Databasus itself</a> or, at least, to copy secret key used for encryption (30 seconds is needed). So you are able to restore from your encrypted backups if you lose access to the server with Databasus or it is corrupted.
---
## 📝 License
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details
---
## 🤝 Contributing
Contributions are welcome! Read the <a href="https://databasus.com/contribute">contributing guide</a> for more details, priorities and rules. If you want to contribute but don't know where to start, message me on Telegram [@rostislav_dugin](https://t.me/rostislav_dugin)
Also you can join our large community of developers, DBAs and DevOps engineers on Telegram [@databasus_community](https://t.me/databasus_community).
--
## 📖 Migration guide
Databasus is the new name for Postgresus. You can stay with latest version of Postgresus if you wish. If you want to migrate - follow installation steps for Databasus itself.
Just renaming an image is not enough as Postgresus and Databasus use different data folders and internal database naming.
You can put a new Databasus image with updated volume near the old Postgresus and run it (stop Postgresus before):
```
services:
databasus:
container_name: databasus
image: databasus/databasus:latest
ports:
- "4005:4005"
volumes:
- ./databasus-data:/databasus-data
restart: unless-stopped
```
Then manually move databases from Postgresus to Databasus.
### Why was Postgresus renamed to Databasus?
Databasus has been developed since 2023. It was internal tool to backup production and home projects databases. In start of 2025 it was released as open source project on GitHub. By the end of 2025 it became popular and the time for renaming has come in December 2025.
It was an important step for the project to grow. Actually, there are a couple of reasons:
1. Postgresus is no longer a little tool that just adds UI for pg_dump for little projects. It became a tool both for individual users, DevOps, DBAs, teams, companies and even large enterprises. Tens of thousands of users use Postgresus every day. Postgresus grew into a reliable backup management tool. Initial positioning is no longer suitable: the project is not just a UI wrapper, it's a solid backup management system now (despite it's still easy to use).
2. New databases are supported: although the primary focus is PostgreSQL (with 100% support in the most efficient way) and always will be, Databasus added support for MySQL, MariaDB and MongoDB. Later more databases will be supported.
3. Trademark issue: "postgres" is a trademark of PostgreSQL Inc. and cannot be used in the project name. So for safety and legal reasons, we had to rename the project.
## AI disclaimer
There have been questions about AI usage in project development in issues and discussions. As the project focuses on security, reliability and production usage, it's important to explain how AI is used in the development process.

View File

@@ -124,6 +124,7 @@ type EnvVariables struct {
SMTPPort int `env:"SMTP_PORT"`
SMTPUser string `env:"SMTP_USER"`
SMTPPassword string `env:"SMTP_PASSWORD"`
SMTPFrom string `env:"SMTP_FROM"`
// Application URL (optional) - used for email links
DatabasusURL string `env:"DATABASUS_URL"`

View File

@@ -1,7 +1,9 @@
package backuping
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
@@ -263,7 +265,7 @@ func (n *BackuperNode) MakeBackup(backupID uuid.UUID, isCallNotifier bool) {
// Delete partial backup from storage
storage, storageErr := n.storageService.GetStorageByID(backup.StorageID)
if storageErr == nil {
if deleteErr := storage.DeleteFile(n.fieldEncryptor, backup.ID.String()); deleteErr != nil {
if deleteErr := storage.DeleteFile(n.fieldEncryptor, backup.FileName); deleteErr != nil {
n.logger.Error(
"Failed to delete partial backup file",
"backupId",
@@ -311,6 +313,13 @@ func (n *BackuperNode) MakeBackup(backupID uuid.UUID, isCallNotifier bool) {
// Update backup with encryption metadata if provided
if backupMetadata != nil {
backupMetadata.BackupID = backup.ID
if err := backupMetadata.Validate(); err != nil {
n.logger.Error("Failed to validate backup metadata", "error", err)
return
}
backup.EncryptionSalt = backupMetadata.EncryptionSalt
backup.EncryptionIV = backupMetadata.EncryptionIV
backup.Encryption = backupMetadata.Encryption
@@ -321,6 +330,39 @@ func (n *BackuperNode) MakeBackup(backupID uuid.UUID, isCallNotifier bool) {
return
}
// Save metadata file to storage
if backupMetadata != nil {
metadataJSON, err := json.Marshal(backupMetadata)
if err != nil {
n.logger.Error("Failed to marshal backup metadata to JSON",
"backupId", backup.ID,
"error", err,
)
} else {
metadataReader := bytes.NewReader(metadataJSON)
metadataFileName := backup.FileName + ".metadata"
if err := storage.SaveFile(
context.Background(),
n.fieldEncryptor,
n.logger,
metadataFileName,
metadataReader,
); err != nil {
n.logger.Error("Failed to save backup metadata file to storage",
"backupId", backup.ID,
"fileName", metadataFileName,
"error", err,
)
} else {
n.logger.Info("Backup metadata file saved successfully",
"backupId", backup.ID,
"fileName", metadataFileName,
)
}
}
}
// Update database last backup time
now := time.Now().UTC()
if updateErr := n.databaseService.SetLastBackupTime(databaseID, now); updateErr != nil {

View File

@@ -18,7 +18,8 @@ import (
)
const (
cleanerTickerInterval = 1 * time.Minute
cleanerTickerInterval = 1 * time.Minute
recentBackupGracePeriod = 60 * time.Minute
)
type BackupCleaner struct {
@@ -51,8 +52,8 @@ func (c *BackupCleaner) Run(ctx context.Context) {
case <-ctx.Done():
return
case <-ticker.C:
if err := c.cleanOldBackups(); err != nil {
c.logger.Error("Failed to clean old backups", "error", err)
if err := c.cleanByRetentionPolicy(); err != nil {
c.logger.Error("Failed to clean backups by retention policy", "error", err)
}
if err := c.cleanExceededBackups(); err != nil {
@@ -79,7 +80,7 @@ func (c *BackupCleaner) DeleteBackup(backup *backups_core.Backup) error {
return err
}
err = storage.DeleteFile(c.fieldEncryptor, backup.ID.String())
err = storage.DeleteFile(c.fieldEncryptor, backup.FileName)
if err != nil {
// we do not return error here, because sometimes clean up performed
// before unavailable storage removal or change - therefore we should
@@ -88,6 +89,11 @@ func (c *BackupCleaner) DeleteBackup(backup *backups_core.Backup) error {
c.logger.Error("Failed to delete backup file", "error", err)
}
metadataFileName := backup.FileName + ".metadata"
if err := storage.DeleteFile(c.fieldEncryptor, metadataFileName); err != nil {
c.logger.Error("Failed to delete backup metadata file", "error", err)
}
return c.backupRepository.DeleteByID(backup.ID)
}
@@ -95,49 +101,30 @@ func (c *BackupCleaner) AddBackupRemoveListener(listener backups_core.BackupRemo
c.backupRemoveListeners = append(c.backupRemoveListeners, listener)
}
func (c *BackupCleaner) cleanOldBackups() error {
func (c *BackupCleaner) cleanByRetentionPolicy() error {
enabledBackupConfigs, err := c.backupConfigService.GetBackupConfigsWithEnabledBackups()
if err != nil {
return err
}
for _, backupConfig := range enabledBackupConfigs {
backupStorePeriod := backupConfig.StorePeriod
var cleanErr error
if backupStorePeriod == period.PeriodForever {
continue
switch backupConfig.RetentionPolicyType {
case backups_config.RetentionPolicyTypeCount:
cleanErr = c.cleanByCount(backupConfig)
case backups_config.RetentionPolicyTypeGFS:
cleanErr = c.cleanByGFS(backupConfig)
default:
cleanErr = c.cleanByTimePeriod(backupConfig)
}
storeDuration := backupStorePeriod.ToDuration()
dateBeforeBackupsShouldBeDeleted := time.Now().UTC().Add(-storeDuration)
oldBackups, err := c.backupRepository.FindBackupsBeforeDate(
backupConfig.DatabaseID,
dateBeforeBackupsShouldBeDeleted,
)
if err != nil {
if cleanErr != nil {
c.logger.Error(
"Failed to find old backups for database",
"databaseId",
backupConfig.DatabaseID,
"error",
err,
)
continue
}
for _, backup := range oldBackups {
if err := c.DeleteBackup(backup); err != nil {
c.logger.Error("Failed to delete old backup", "backupId", backup.ID, "error", err)
continue
}
c.logger.Info(
"Deleted old backup",
"backupId",
backup.ID,
"databaseId",
backupConfig.DatabaseID,
"Failed to clean backups by retention policy",
"databaseId", backupConfig.DatabaseID,
"policy", backupConfig.RetentionPolicyType,
"error", cleanErr,
)
}
}
@@ -174,6 +161,158 @@ func (c *BackupCleaner) cleanExceededBackups() error {
return nil
}
func (c *BackupCleaner) cleanByTimePeriod(backupConfig *backups_config.BackupConfig) error {
if backupConfig.RetentionTimePeriod == "" {
return nil
}
if backupConfig.RetentionTimePeriod == period.PeriodForever {
return nil
}
storeDuration := backupConfig.RetentionTimePeriod.ToDuration()
dateBeforeBackupsShouldBeDeleted := time.Now().UTC().Add(-storeDuration)
oldBackups, err := c.backupRepository.FindBackupsBeforeDate(
backupConfig.DatabaseID,
dateBeforeBackupsShouldBeDeleted,
)
if err != nil {
return fmt.Errorf(
"failed to find old backups for database %s: %w",
backupConfig.DatabaseID,
err,
)
}
for _, backup := range oldBackups {
if isRecentBackup(backup) {
continue
}
if err := c.DeleteBackup(backup); err != nil {
c.logger.Error("Failed to delete old backup", "backupId", backup.ID, "error", err)
continue
}
c.logger.Info(
"Deleted old backup",
"backupId", backup.ID,
"databaseId", backupConfig.DatabaseID,
)
}
return nil
}
func (c *BackupCleaner) cleanByCount(backupConfig *backups_config.BackupConfig) error {
if backupConfig.RetentionCount <= 0 {
return nil
}
completedBackups, err := c.backupRepository.FindByDatabaseIdAndStatus(
backupConfig.DatabaseID,
backups_core.BackupStatusCompleted,
)
if err != nil {
return fmt.Errorf(
"failed to find completed backups for database %s: %w",
backupConfig.DatabaseID,
err,
)
}
// completedBackups are ordered newest first; delete everything beyond position RetentionCount
if len(completedBackups) <= backupConfig.RetentionCount {
return nil
}
toDelete := completedBackups[backupConfig.RetentionCount:]
for _, backup := range toDelete {
if isRecentBackup(backup) {
continue
}
if err := c.DeleteBackup(backup); err != nil {
c.logger.Error(
"Failed to delete backup by count policy",
"backupId",
backup.ID,
"error",
err,
)
continue
}
c.logger.Info(
"Deleted backup by count policy",
"backupId", backup.ID,
"databaseId", backupConfig.DatabaseID,
"retentionCount", backupConfig.RetentionCount,
)
}
return nil
}
func (c *BackupCleaner) cleanByGFS(backupConfig *backups_config.BackupConfig) error {
if backupConfig.RetentionGfsHours <= 0 && backupConfig.RetentionGfsDays <= 0 &&
backupConfig.RetentionGfsWeeks <= 0 && backupConfig.RetentionGfsMonths <= 0 &&
backupConfig.RetentionGfsYears <= 0 {
return nil
}
completedBackups, err := c.backupRepository.FindByDatabaseIdAndStatus(
backupConfig.DatabaseID,
backups_core.BackupStatusCompleted,
)
if err != nil {
return fmt.Errorf(
"failed to find completed backups for database %s: %w",
backupConfig.DatabaseID,
err,
)
}
keepSet := buildGFSKeepSet(
completedBackups,
backupConfig.RetentionGfsHours,
backupConfig.RetentionGfsDays,
backupConfig.RetentionGfsWeeks,
backupConfig.RetentionGfsMonths,
backupConfig.RetentionGfsYears,
)
for _, backup := range completedBackups {
if keepSet[backup.ID] {
continue
}
if isRecentBackup(backup) {
continue
}
if err := c.DeleteBackup(backup); err != nil {
c.logger.Error(
"Failed to delete backup by GFS policy",
"backupId",
backup.ID,
"error",
err,
)
continue
}
c.logger.Info(
"Deleted backup by GFS policy",
"backupId", backup.ID,
"databaseId", backupConfig.DatabaseID,
)
}
return nil
}
func (c *BackupCleaner) cleanExceededBackupsForDatabase(
databaseID uuid.UUID,
limitperDbMB int64,
@@ -210,6 +349,21 @@ func (c *BackupCleaner) cleanExceededBackupsForDatabase(
}
backup := oldestBackups[0]
if isRecentBackup(backup) {
c.logger.Warn(
"Oldest backup is too recent to delete, stopping size cleanup",
"databaseId",
databaseID,
"backupId",
backup.ID,
"totalSizeMB",
backupsTotalSizeMB,
"limitMB",
limitperDbMB,
)
break
}
if err := c.DeleteBackup(backup); err != nil {
c.logger.Error(
"Failed to delete exceeded backup",
@@ -240,3 +394,68 @@ func (c *BackupCleaner) cleanExceededBackupsForDatabase(
return nil
}
func isRecentBackup(backup *backups_core.Backup) bool {
return time.Since(backup.CreatedAt) < recentBackupGracePeriod
}
// buildGFSKeepSet determines which backups to retain under the GFS rotation scheme.
// Backups must be sorted newest-first. A backup can fill multiple slots simultaneously
// (e.g. the newest backup of a year also fills the monthly, weekly, daily, and hourly slot).
func buildGFSKeepSet(
backups []*backups_core.Backup,
hours, days, weeks, months, years int,
) map[uuid.UUID]bool {
keep := make(map[uuid.UUID]bool)
hoursSeen := make(map[string]bool)
daysSeen := make(map[string]bool)
weeksSeen := make(map[string]bool)
monthsSeen := make(map[string]bool)
yearsSeen := make(map[string]bool)
hoursKept, daysKept, weeksKept, monthsKept, yearsKept := 0, 0, 0, 0, 0
for _, backup := range backups {
t := backup.CreatedAt
hourKey := t.Format("2006-01-02-15")
dayKey := t.Format("2006-01-02")
weekYear, week := t.ISOWeek()
weekKey := fmt.Sprintf("%d-%02d", weekYear, week)
monthKey := t.Format("2006-01")
yearKey := t.Format("2006")
if hours > 0 && hoursKept < hours && !hoursSeen[hourKey] {
keep[backup.ID] = true
hoursSeen[hourKey] = true
hoursKept++
}
if days > 0 && daysKept < days && !daysSeen[dayKey] {
keep[backup.ID] = true
daysSeen[dayKey] = true
daysKept++
}
if weeks > 0 && weeksKept < weeks && !weeksSeen[weekKey] {
keep[backup.ID] = true
weeksSeen[weekKey] = true
weeksKept++
}
if months > 0 && monthsKept < months && !monthsSeen[monthKey] {
keep[backup.ID] = true
monthsSeen[monthKey] = true
monthsKept++
}
if years > 0 && yearsKept < years && !yearsSeen[yearKey] {
keep[backup.ID] = true
yearsSeen[yearKey] = true
yearsKept++
}
}
return keep
}

View File

@@ -57,7 +57,8 @@ func Test_RunPendingBackups_WhenLastBackupWasYesterday_CreatesNewBackup(t *testi
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
@@ -126,7 +127,8 @@ func Test_RunPendingBackups_WhenLastBackupWasRecentlyCompleted_SkipsBackup(t *te
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
@@ -194,7 +196,8 @@ func Test_RunPendingBackups_WhenLastBackupFailedAndRetriesDisabled_SkipsBackup(t
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
backupConfig.IsRetryIfFailed = false
@@ -266,7 +269,8 @@ func Test_RunPendingBackups_WhenLastBackupFailedAndRetriesEnabled_CreatesNewBack
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
backupConfig.IsRetryIfFailed = true
@@ -339,7 +343,8 @@ func Test_RunPendingBackups_WhenFailedBackupsExceedMaxRetries_SkipsBackup(t *tes
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
backupConfig.IsRetryIfFailed = true
@@ -410,7 +415,8 @@ func Test_RunPendingBackups_WhenBackupsDisabled_SkipsBackup(t *testing.T) {
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = false
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
@@ -479,7 +485,8 @@ func Test_CheckDeadNodesAndFailBackups_WhenNodeDies_FailsBackupAndCleansUpRegist
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
@@ -582,7 +589,8 @@ func Test_OnBackupCompleted_WhenTaskIsNotBackup_SkipsProcessing(t *testing.T) {
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
@@ -759,7 +767,8 @@ func Test_FailBackupsInProgress_WhenSchedulerStarts_CancelsBackupsAndUpdatesStat
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
@@ -872,7 +881,8 @@ func Test_StartBackup_WhenBackupCompletes_DecrementsActiveTaskCount(t *testing.T
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
@@ -975,7 +985,8 @@ func Test_StartBackup_WhenBackupFails_DecrementsActiveTaskCount(t *testing.T) {
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
@@ -1069,7 +1080,8 @@ func Test_StartBackup_WhenBackupAlreadyInProgress_SkipsNewBackup(t *testing.T) {
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
@@ -1140,7 +1152,8 @@ func Test_RunPendingBackups_WhenLastBackupFailedWithIsSkipRetry_SkipsBackupEvenW
TimeOfDay: &timeOfDay,
}
backupConfig.IsBackupsEnabled = true
backupConfig.StorePeriod = period.PeriodWeek
backupConfig.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig.RetentionTimePeriod = period.PeriodWeek
backupConfig.Storage = storage
backupConfig.StorageID = &storage.ID
backupConfig.IsRetryIfFailed = true
@@ -1242,7 +1255,8 @@ func Test_StartBackup_When2BackupsStartedForDifferentDatabases_BothUseCasesAreCa
TimeOfDay: &timeOfDay,
}
backupConfig1.IsBackupsEnabled = true
backupConfig1.StorePeriod = period.PeriodWeek
backupConfig1.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig1.RetentionTimePeriod = period.PeriodWeek
backupConfig1.Storage = storage
backupConfig1.StorageID = &storage.ID
@@ -1259,7 +1273,8 @@ func Test_StartBackup_When2BackupsStartedForDifferentDatabases_BothUseCasesAreCa
TimeOfDay: &timeOfDay,
}
backupConfig2.IsBackupsEnabled = true
backupConfig2.StorePeriod = period.PeriodWeek
backupConfig2.RetentionPolicyType = backups_config.RetentionPolicyTypeTimePeriod
backupConfig2.RetentionTimePeriod = period.PeriodWeek
backupConfig2.Storage = storage
backupConfig2.StorageID = &storage.ID

View File

@@ -1,17 +1,37 @@
package common
import backups_config "databasus-backend/internal/features/backups/config"
import (
backups_config "databasus-backend/internal/features/backups/config"
"errors"
type BackupType string
const (
BackupTypeDefault BackupType = "DEFAULT" // For MySQL, MongoDB, PostgreSQL legacy (-Fc)
BackupTypeDirectory BackupType = "DIRECTORY" // PostgreSQL directory type (-Fd)
"github.com/google/uuid"
)
type BackupMetadata struct {
EncryptionSalt *string
EncryptionIV *string
Encryption backups_config.BackupEncryption
Type BackupType
BackupID uuid.UUID `json:"backupId"`
EncryptionSalt *string `json:"encryptionSalt"`
EncryptionIV *string `json:"encryptionIV"`
Encryption backups_config.BackupEncryption `json:"encryption"`
}
func (m *BackupMetadata) Validate() error {
if m.BackupID == uuid.Nil {
return errors.New("backup ID is required")
}
if m.Encryption == "" {
return errors.New("encryption is required")
}
if m.Encryption == backups_config.BackupEncryptionEncrypted {
if m.EncryptionSalt == nil {
return errors.New("encryption salt is required when encryption is enabled")
}
if m.EncryptionIV == nil {
return errors.New("encryption IV is required when encryption is enabled")
}
}
return nil
}

View File

@@ -305,7 +305,6 @@ func (c *BackupController) GetFile(ctx *gin.Context) {
_, err = io.Copy(ctx.Writer, rateLimitedReader)
if err != nil {
fmt.Printf("Error streaming file: %v\n", err)
return
}
c.backupService.WriteAuditLogForDownload(downloadToken.UserID, backup, database)

View File

@@ -7,6 +7,8 @@ import (
"io"
"log/slog"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
@@ -18,6 +20,8 @@ import (
"databasus-backend/internal/config"
audit_logs "databasus-backend/internal/features/audit_logs"
"databasus-backend/internal/features/backups/backups/backuping"
backups_common "databasus-backend/internal/features/backups/backups/common"
backups_core "databasus-backend/internal/features/backups/backups/core"
backups_download "databasus-backend/internal/features/backups/backups/download"
backups_config "databasus-backend/internal/features/backups/config"
@@ -1245,6 +1249,86 @@ func Test_GenerateDownloadToken_BlockedWhenDownloadInProgress(t *testing.T) {
workspaces_testing.RemoveTestWorkspace(workspace, router)
}
func Test_MakeBackup_VerifyBackupAndMetadataFilesExistInStorage(t *testing.T) {
router := createTestRouter()
owner := users_testing.CreateTestUser(users_enums.UserRoleMember)
workspace := workspaces_testing.CreateTestWorkspace("Test Workspace", owner, router)
database, _, storage := createTestDatabaseWithBackups(workspace, owner, router)
backuperNode := backuping.CreateTestBackuperNode()
backuperCancel := backuping.StartBackuperNodeForTest(t, backuperNode)
defer backuping.StopBackuperNodeForTest(t, backuperCancel, backuperNode)
scheduler := backuping.CreateTestScheduler()
schedulerCancel := backuping.StartSchedulerForTest(t, scheduler)
defer schedulerCancel()
backupRepo := &backups_core.BackupRepository{}
initialBackups, err := backupRepo.FindByDatabaseID(database.ID)
assert.NoError(t, err)
request := MakeBackupRequest{DatabaseID: database.ID}
test_utils.MakePostRequest(
t,
router,
"/api/v1/backups",
"Bearer "+owner.Token,
request,
http.StatusOK,
)
backuping.WaitForBackupCompletion(t, database.ID, len(initialBackups), 30*time.Second)
backups, err := backupRepo.FindByDatabaseID(database.ID)
assert.NoError(t, err)
assert.Greater(t, len(backups), len(initialBackups))
backup := backups[0]
assert.Equal(t, backups_core.BackupStatusCompleted, backup.Status)
storageService := storages.GetStorageService()
backupStorage, err := storageService.GetStorageByID(backup.StorageID)
assert.NoError(t, err)
encryptor := encryption.GetFieldEncryptor()
backupFile, err := backupStorage.GetFile(encryptor, backup.FileName)
assert.NoError(t, err)
backupFile.Close()
metadataFile, err := backupStorage.GetFile(encryptor, backup.FileName+".metadata")
assert.NoError(t, err)
metadataContent, err := io.ReadAll(metadataFile)
assert.NoError(t, err)
metadataFile.Close()
var storageMetadata backups_common.BackupMetadata
err = json.Unmarshal(metadataContent, &storageMetadata)
assert.NoError(t, err)
assert.Equal(t, backup.ID, storageMetadata.BackupID)
if backup.EncryptionSalt != nil && storageMetadata.EncryptionSalt != nil {
assert.Equal(t, *backup.EncryptionSalt, *storageMetadata.EncryptionSalt)
}
if backup.EncryptionIV != nil && storageMetadata.EncryptionIV != nil {
assert.Equal(t, *backup.EncryptionIV, *storageMetadata.EncryptionIV)
}
assert.Equal(t, backup.Encryption, storageMetadata.Encryption)
err = backupRepo.DeleteByID(backup.ID)
assert.NoError(t, err)
databases.RemoveTestDatabase(database)
time.Sleep(50 * time.Millisecond)
storages.RemoveTestStorage(storage.ID)
workspaces_testing.RemoveTestWorkspace(workspace, router)
}
func createTestRouter() *gin.Engine {
return CreateTestRouter()
}
@@ -1721,3 +1805,84 @@ func Test_BandwidthThrottling_DynamicAdjustment(t *testing.T) {
storages.RemoveTestStorage(storage.ID)
workspaces_testing.RemoveTestWorkspace(workspace, router)
}
func Test_DeleteBackup_RemovesBackupAndMetadataFilesFromDisk(t *testing.T) {
router := createTestRouter()
owner := users_testing.CreateTestUser(users_enums.UserRoleMember)
workspace := workspaces_testing.CreateTestWorkspace("Test Workspace", owner, router)
database := createTestDatabase("Test Database", workspace.ID, owner.Token, router)
storage := createTestStorage(workspace.ID)
configService := backups_config.GetBackupConfigService()
backupConfig, err := configService.GetBackupConfigByDbId(database.ID)
assert.NoError(t, err)
backupConfig.IsBackupsEnabled = true
backupConfig.StorageID = &storage.ID
backupConfig.Storage = storage
_, err = configService.SaveBackupConfig(backupConfig)
assert.NoError(t, err)
defer func() {
databases.RemoveTestDatabase(database)
time.Sleep(50 * time.Millisecond)
storages.RemoveTestStorage(storage.ID)
workspaces_testing.RemoveTestWorkspace(workspace, router)
}()
backuperNode := backuping.CreateTestBackuperNode()
backuperCancel := backuping.StartBackuperNodeForTest(t, backuperNode)
defer backuping.StopBackuperNodeForTest(t, backuperCancel, backuperNode)
scheduler := backuping.CreateTestScheduler()
schedulerCancel := backuping.StartSchedulerForTest(t, scheduler)
defer schedulerCancel()
backupRepo := &backups_core.BackupRepository{}
initialBackups, err := backupRepo.FindByDatabaseID(database.ID)
assert.NoError(t, err)
request := MakeBackupRequest{DatabaseID: database.ID}
test_utils.MakePostRequest(
t,
router,
"/api/v1/backups",
"Bearer "+owner.Token,
request,
http.StatusOK,
)
backuping.WaitForBackupCompletion(t, database.ID, len(initialBackups), 30*time.Second)
backups, err := backupRepo.FindByDatabaseID(database.ID)
assert.NoError(t, err)
assert.Greater(t, len(backups), len(initialBackups))
backup := backups[0]
assert.Equal(t, backups_core.BackupStatusCompleted, backup.Status)
dataFolder := config.GetEnv().DataFolder
backupFilePath := filepath.Join(dataFolder, backup.FileName)
metadataFilePath := filepath.Join(dataFolder, backup.FileName+".metadata")
_, err = os.Stat(backupFilePath)
assert.NoError(t, err, "backup file should exist on disk before deletion")
_, err = os.Stat(metadataFilePath)
assert.NoError(t, err, "metadata file should exist on disk before deletion")
test_utils.MakeDeleteRequest(
t,
router,
fmt.Sprintf("/api/v1/backups/%s", backup.ID.String()),
"Bearer "+owner.Token,
http.StatusNoContent,
)
_, err = os.Stat(backupFilePath)
assert.True(t, os.IsNotExist(err), "backup file should be removed from disk after deletion")
_, err = os.Stat(metadataFilePath)
assert.True(t, os.IsNotExist(err), "metadata file should be removed from disk after deletion")
}

View File

@@ -325,7 +325,7 @@ func (s *BackupService) getBackupReader(backupID uuid.UUID) (io.ReadCloser, erro
return nil, fmt.Errorf("failed to get storage: %w", err)
}
fileReader, err := storage.GetFile(s.fieldEncryptor, backup.ID.String())
fileReader, err := storage.GetFile(s.fieldEncryptor, backup.FileName)
if err != nil {
return nil, fmt.Errorf("failed to get backup file: %w", err)
}

View File

@@ -427,7 +427,9 @@ func (uc *CreateMariadbBackupUsecase) setupBackupEncryption(
backupConfig *backups_config.BackupConfig,
storageWriter io.WriteCloser,
) (io.Writer, *backup_encryption.EncryptionWriter, common.BackupMetadata, error) {
metadata := common.BackupMetadata{}
metadata := common.BackupMetadata{
BackupID: backupID,
}
if backupConfig.Encryption != backups_config.BackupEncryptionEncrypted {
metadata.Encryption = backups_config.BackupEncryptionNone

View File

@@ -269,6 +269,7 @@ func (uc *CreateMongodbBackupUsecase) setupBackupEncryption(
storageWriter io.WriteCloser,
) (io.Writer, *backup_encryption.EncryptionWriter, common.BackupMetadata, error) {
backupMetadata := common.BackupMetadata{
BackupID: backupID,
Encryption: backups_config.BackupEncryptionNone,
}
@@ -305,6 +306,7 @@ func (uc *CreateMongodbBackupUsecase) setupBackupEncryption(
saltBase64 := base64.StdEncoding.EncodeToString(salt)
nonceBase64 := base64.StdEncoding.EncodeToString(nonce)
backupMetadata.BackupID = backupID
backupMetadata.Encryption = backups_config.BackupEncryptionEncrypted
backupMetadata.EncryptionSalt = &saltBase64
backupMetadata.EncryptionIV = &nonceBase64

View File

@@ -438,7 +438,9 @@ func (uc *CreateMysqlBackupUsecase) setupBackupEncryption(
backupConfig *backups_config.BackupConfig,
storageWriter io.WriteCloser,
) (io.Writer, *backup_encryption.EncryptionWriter, common.BackupMetadata, error) {
metadata := common.BackupMetadata{}
metadata := common.BackupMetadata{
BackupID: backupID,
}
if backupConfig.Encryption != backups_config.BackupEncryptionEncrypted {
metadata.Encryption = backups_config.BackupEncryptionNone

View File

@@ -482,7 +482,9 @@ func (uc *CreatePostgresqlBackupUsecase) setupBackupEncryption(
backupConfig *backups_config.BackupConfig,
storageWriter io.WriteCloser,
) (io.Writer, *backup_encryption.EncryptionWriter, common.BackupMetadata, error) {
metadata := common.BackupMetadata{}
metadata := common.BackupMetadata{
BackupID: backupID,
}
if backupConfig.Encryption != backups_config.BackupEncryptionEncrypted {
metadata.Encryption = backups_config.BackupEncryptionNone

View File

@@ -118,9 +118,10 @@ func Test_SaveBackupConfig_PermissionsEnforced(t *testing.T) {
timeOfDay := "04:00"
request := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -146,7 +147,7 @@ func Test_SaveBackupConfig_PermissionsEnforced(t *testing.T) {
if tt.expectSuccess {
assert.Equal(t, database.ID, response.DatabaseID)
assert.True(t, response.IsBackupsEnabled)
assert.Equal(t, period.PeriodWeek, response.StorePeriod)
assert.Equal(t, period.PeriodWeek, response.RetentionTimePeriod)
} else {
assert.Contains(t, string(testResp.Body), "insufficient permissions")
}
@@ -170,9 +171,10 @@ func Test_SaveBackupConfig_WhenUserIsNotWorkspaceMember_ReturnsForbidden(t *test
timeOfDay := "04:00"
request := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -337,7 +339,7 @@ func Test_GetBackupConfigByDbID_ReturnsDefaultConfigForNewDatabase(t *testing.T)
assert.Equal(t, database.ID, response.DatabaseID)
assert.False(t, response.IsBackupsEnabled)
assert.Equal(t, plan.MaxStoragePeriod, response.StorePeriod)
assert.Equal(t, plan.MaxStoragePeriod, response.RetentionTimePeriod)
assert.Equal(t, plan.MaxBackupSizeMB, response.MaxBackupSizeMB)
assert.Equal(t, plan.MaxBackupsTotalSizeMB, response.MaxBackupsTotalSizeMB)
assert.True(t, response.IsRetryIfFailed)
@@ -411,9 +413,10 @@ func Test_SaveBackupConfig_WhenPlanLimitsAreAdjusted_ValidationEnforced(t *testi
// Test 1: Try to save backup config with exceeded backup size limit
timeOfDay := "04:00"
backupConfigExceededSize := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -440,9 +443,10 @@ func Test_SaveBackupConfig_WhenPlanLimitsAreAdjusted_ValidationEnforced(t *testi
// Test 2: Try to save backup config with exceeded total size limit
backupConfigExceededTotal := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -469,9 +473,10 @@ func Test_SaveBackupConfig_WhenPlanLimitsAreAdjusted_ValidationEnforced(t *testi
// Test 3: Try to save backup config with exceeded storage period limit
backupConfigExceededPeriod := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodYear, // Exceeds limit of Month
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodYear, // Exceeds limit of Month
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -498,9 +503,10 @@ func Test_SaveBackupConfig_WhenPlanLimitsAreAdjusted_ValidationEnforced(t *testi
// Test 4: Save backup config within all limits - should succeed
backupConfigValid := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek, // Within Month limit
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek, // Within Month limit
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -529,7 +535,7 @@ func Test_SaveBackupConfig_WhenPlanLimitsAreAdjusted_ValidationEnforced(t *testi
assert.Equal(t, database.ID, responseValid.DatabaseID)
assert.Equal(t, int64(80), responseValid.MaxBackupSizeMB)
assert.Equal(t, int64(800), responseValid.MaxBackupsTotalSizeMB)
assert.Equal(t, period.PeriodWeek, responseValid.StorePeriod)
assert.Equal(t, period.PeriodWeek, responseValid.RetentionTimePeriod)
}
func Test_IsStorageUsing_PermissionsEnforced(t *testing.T) {
@@ -618,9 +624,10 @@ func Test_SaveBackupConfig_WithEncryptionNone_ConfigSaved(t *testing.T) {
timeOfDay := "04:00"
request := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -662,9 +669,10 @@ func Test_SaveBackupConfig_WithEncryptionEncrypted_ConfigSaved(t *testing.T) {
timeOfDay := "04:00"
request := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -959,9 +967,10 @@ func Test_TransferDatabase_ToNewStorage_DatabaseTransferd(t *testing.T) {
timeOfDay := "04:00"
backupConfigRequest := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1045,9 +1054,10 @@ func Test_TransferDatabase_WithExistingStorage_DatabaseAndStorageTransferd(t *te
timeOfDay := "04:00"
backupConfigRequest := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1142,9 +1152,10 @@ func Test_TransferDatabase_StorageHasOtherDBs_CannotTransfer(t *testing.T) {
timeOfDay := "04:00"
backupConfigRequest1 := BackupConfig{
DatabaseID: database1.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database1.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1168,9 +1179,10 @@ func Test_TransferDatabase_StorageHasOtherDBs_CannotTransfer(t *testing.T) {
)
backupConfigRequest2 := BackupConfig{
DatabaseID: database2.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database2.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1244,9 +1256,10 @@ func Test_TransferDatabase_WithNotifiers_NotifiersTransferred(t *testing.T) {
timeOfDay := "04:00"
backupConfigRequest := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1364,9 +1377,10 @@ func Test_TransferDatabase_NotifierHasOtherDBs_NotifierSkipped(t *testing.T) {
timeOfDay := "04:00"
backupConfigRequest := BackupConfig{
DatabaseID: database1.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database1.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1486,9 +1500,10 @@ func Test_TransferDatabase_WithMultipleNotifiers_OnlyExclusiveOnesTransferred(t
timeOfDay := "04:00"
backupConfigRequest := BackupConfig{
DatabaseID: database1.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database1.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1585,9 +1600,10 @@ func Test_TransferDatabase_WithTargetNotifiers_NotifiersAssigned(t *testing.T) {
timeOfDay := "04:00"
backupConfigRequest := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1665,9 +1681,10 @@ func Test_TransferDatabase_TargetNotifierFromDifferentWorkspace_ReturnsBadReques
timeOfDay := "04:00"
backupConfigRequest := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1730,9 +1747,10 @@ func Test_TransferDatabase_TargetStorageFromDifferentWorkspace_ReturnsBadRequest
timeOfDay := "04:00"
backupConfigRequest := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1789,9 +1807,10 @@ func Test_SaveBackupConfig_WithSystemStorage_CanBeUsedByAnyDatabase(t *testing.T
timeOfDay := "04:00"
backupConfigWithRegularStorage := BackupConfig{
DatabaseID: databaseA.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: databaseA.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -1840,9 +1859,10 @@ func Test_SaveBackupConfig_WithSystemStorage_CanBeUsedByAnyDatabase(t *testing.T
assert.True(t, savedSystemStorage.IsSystem)
backupConfigWithSystemStorage := BackupConfig{
DatabaseID: databaseA.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: databaseA.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,

View File

@@ -13,3 +13,11 @@ const (
BackupEncryptionNone BackupEncryption = "NONE"
BackupEncryptionEncrypted BackupEncryption = "ENCRYPTED"
)
type RetentionPolicyType string
const (
RetentionPolicyTypeTimePeriod RetentionPolicyType = "TIME_PERIOD"
RetentionPolicyTypeCount RetentionPolicyType = "COUNT"
RetentionPolicyTypeGFS RetentionPolicyType = "GFS"
)

View File

@@ -18,7 +18,15 @@ type BackupConfig struct {
IsBackupsEnabled bool `json:"isBackupsEnabled" gorm:"column:is_backups_enabled;type:boolean;not null"`
StorePeriod period.Period `json:"storePeriod" gorm:"column:store_period;type:text;not null"`
RetentionPolicyType RetentionPolicyType `json:"retentionPolicyType" gorm:"column:retention_policy_type;type:text;not null;default:'TIME_PERIOD'"`
RetentionTimePeriod period.TimePeriod `json:"retentionTimePeriod" gorm:"column:retention_time_period;type:text;not null;default:''"`
RetentionCount int `json:"retentionCount" gorm:"column:retention_count;type:int;not null;default:0"`
RetentionGfsHours int `json:"retentionGfsHours" gorm:"column:retention_gfs_hours;type:int;not null;default:0"`
RetentionGfsDays int `json:"retentionGfsDays" gorm:"column:retention_gfs_days;type:int;not null;default:0"`
RetentionGfsWeeks int `json:"retentionGfsWeeks" gorm:"column:retention_gfs_weeks;type:int;not null;default:0"`
RetentionGfsMonths int `json:"retentionGfsMonths" gorm:"column:retention_gfs_months;type:int;not null;default:0"`
RetentionGfsYears int `json:"retentionGfsYears" gorm:"column:retention_gfs_years;type:int;not null;default:0"`
BackupIntervalID uuid.UUID `json:"backupIntervalId" gorm:"column:backup_interval_id;type:uuid;not null"`
BackupInterval *intervals.Interval `json:"backupInterval,omitempty" gorm:"foreignKey:BackupIntervalID"`
@@ -78,13 +86,12 @@ func (b *BackupConfig) AfterFind(tx *gorm.DB) error {
}
func (b *BackupConfig) Validate(plan *plans.DatabasePlan) error {
// Backup interval is required either as ID or as object
if b.BackupIntervalID == uuid.Nil && b.BackupInterval == nil {
return errors.New("backup interval is required")
}
if b.StorePeriod == "" {
return errors.New("store period is required")
if err := b.validateRetentionPolicy(plan); err != nil {
return err
}
if b.IsRetryIfFailed && b.MaxFailedTriesCount <= 0 {
@@ -110,22 +117,12 @@ func (b *BackupConfig) Validate(plan *plans.DatabasePlan) error {
return errors.New("max backups total size must be non-negative")
}
// Validate against plan limits
// Check storage period limit
if plan.MaxStoragePeriod != period.PeriodForever {
if b.StorePeriod.CompareTo(plan.MaxStoragePeriod) > 0 {
return errors.New("storage period exceeds plan limit")
}
}
// Check max backup size limit (0 in plan means unlimited)
if plan.MaxBackupSizeMB > 0 {
if b.MaxBackupSizeMB == 0 || b.MaxBackupSizeMB > plan.MaxBackupSizeMB {
return errors.New("max backup size exceeds plan limit")
}
}
// Check max total backups size limit (0 in plan means unlimited)
if plan.MaxBackupsTotalSizeMB > 0 {
if b.MaxBackupsTotalSizeMB == 0 ||
b.MaxBackupsTotalSizeMB > plan.MaxBackupsTotalSizeMB {
@@ -140,7 +137,14 @@ func (b *BackupConfig) Copy(newDatabaseID uuid.UUID) *BackupConfig {
return &BackupConfig{
DatabaseID: newDatabaseID,
IsBackupsEnabled: b.IsBackupsEnabled,
StorePeriod: b.StorePeriod,
RetentionPolicyType: b.RetentionPolicyType,
RetentionTimePeriod: b.RetentionTimePeriod,
RetentionCount: b.RetentionCount,
RetentionGfsHours: b.RetentionGfsHours,
RetentionGfsDays: b.RetentionGfsDays,
RetentionGfsWeeks: b.RetentionGfsWeeks,
RetentionGfsMonths: b.RetentionGfsMonths,
RetentionGfsYears: b.RetentionGfsYears,
BackupIntervalID: uuid.Nil,
BackupInterval: b.BackupInterval.Copy(),
StorageID: b.StorageID,
@@ -152,3 +156,34 @@ func (b *BackupConfig) Copy(newDatabaseID uuid.UUID) *BackupConfig {
MaxBackupsTotalSizeMB: b.MaxBackupsTotalSizeMB,
}
}
func (b *BackupConfig) validateRetentionPolicy(plan *plans.DatabasePlan) error {
switch b.RetentionPolicyType {
case RetentionPolicyTypeTimePeriod, "":
if b.RetentionTimePeriod == "" {
return errors.New("retention time period is required")
}
if plan.MaxStoragePeriod != period.PeriodForever {
if b.RetentionTimePeriod.CompareTo(plan.MaxStoragePeriod) > 0 {
return errors.New("storage period exceeds plan limit")
}
}
case RetentionPolicyTypeCount:
if b.RetentionCount <= 0 {
return errors.New("retention count must be greater than 0")
}
case RetentionPolicyTypeGFS:
if b.RetentionGfsHours <= 0 && b.RetentionGfsDays <= 0 && b.RetentionGfsWeeks <= 0 &&
b.RetentionGfsMonths <= 0 && b.RetentionGfsYears <= 0 {
return errors.New("at least one GFS retention field must be greater than 0")
}
default:
return errors.New("invalid retention policy type")
}
return nil
}

View File

@@ -11,9 +11,9 @@ import (
"github.com/stretchr/testify/assert"
)
func Test_Validate_WhenStoragePeriodIsWeekAndPlanAllowsMonth_ValidationPasses(t *testing.T) {
func Test_Validate_WhenRetentionTimePeriodIsWeekAndPlanAllowsMonth_ValidationPasses(t *testing.T) {
config := createValidBackupConfig()
config.StorePeriod = period.PeriodWeek
config.RetentionTimePeriod = period.PeriodWeek
plan := createUnlimitedPlan()
plan.MaxStoragePeriod = period.PeriodMonth
@@ -22,9 +22,9 @@ func Test_Validate_WhenStoragePeriodIsWeekAndPlanAllowsMonth_ValidationPasses(t
assert.NoError(t, err)
}
func Test_Validate_WhenStoragePeriodIsYearAndPlanAllowsMonth_ValidationFails(t *testing.T) {
func Test_Validate_WhenRetentionTimePeriodIsYearAndPlanAllowsMonth_ValidationFails(t *testing.T) {
config := createValidBackupConfig()
config.StorePeriod = period.PeriodYear
config.RetentionTimePeriod = period.PeriodYear
plan := createUnlimitedPlan()
plan.MaxStoragePeriod = period.PeriodMonth
@@ -33,9 +33,11 @@ func Test_Validate_WhenStoragePeriodIsYearAndPlanAllowsMonth_ValidationFails(t *
assert.EqualError(t, err, "storage period exceeds plan limit")
}
func Test_Validate_WhenStoragePeriodIsForeverAndPlanAllowsForever_ValidationPasses(t *testing.T) {
func Test_Validate_WhenRetentionTimePeriodIsForeverAndPlanAllowsForever_ValidationPasses(
t *testing.T,
) {
config := createValidBackupConfig()
config.StorePeriod = period.PeriodForever
config.RetentionTimePeriod = period.PeriodForever
plan := createUnlimitedPlan()
plan.MaxStoragePeriod = period.PeriodForever
@@ -44,9 +46,9 @@ func Test_Validate_WhenStoragePeriodIsForeverAndPlanAllowsForever_ValidationPass
assert.NoError(t, err)
}
func Test_Validate_WhenStoragePeriodIsForeverAndPlanAllowsYear_ValidationFails(t *testing.T) {
func Test_Validate_WhenRetentionTimePeriodIsForeverAndPlanAllowsYear_ValidationFails(t *testing.T) {
config := createValidBackupConfig()
config.StorePeriod = period.PeriodForever
config.RetentionTimePeriod = period.PeriodForever
plan := createUnlimitedPlan()
plan.MaxStoragePeriod = period.PeriodYear
@@ -55,9 +57,9 @@ func Test_Validate_WhenStoragePeriodIsForeverAndPlanAllowsYear_ValidationFails(t
assert.EqualError(t, err, "storage period exceeds plan limit")
}
func Test_Validate_WhenStoragePeriodEqualsExactPlanLimit_ValidationPasses(t *testing.T) {
func Test_Validate_WhenRetentionTimePeriodEqualsExactPlanLimit_ValidationPasses(t *testing.T) {
config := createValidBackupConfig()
config.StorePeriod = period.PeriodMonth
config.RetentionTimePeriod = period.PeriodMonth
plan := createUnlimitedPlan()
plan.MaxStoragePeriod = period.PeriodMonth
@@ -178,7 +180,7 @@ func Test_Validate_WhenTotalSizeEqualsExactPlanLimit_ValidationPasses(t *testing
func Test_Validate_WhenAllLimitsAreUnlimitedInPlan_AnyConfigurationPasses(t *testing.T) {
config := createValidBackupConfig()
config.StorePeriod = period.PeriodForever
config.RetentionTimePeriod = period.PeriodForever
config.MaxBackupSizeMB = 0
config.MaxBackupsTotalSizeMB = 0
@@ -190,7 +192,7 @@ func Test_Validate_WhenAllLimitsAreUnlimitedInPlan_AnyConfigurationPasses(t *tes
func Test_Validate_WhenMultipleLimitsExceeded_ValidationFailsWithFirstError(t *testing.T) {
config := createValidBackupConfig()
config.StorePeriod = period.PeriodYear
config.RetentionTimePeriod = period.PeriodYear
config.MaxBackupSizeMB = 500
config.MaxBackupsTotalSizeMB = 5000
@@ -249,14 +251,14 @@ func Test_Validate_WhenEncryptionIsInvalid_ValidationFailsRegardlessOfPlan(t *te
assert.EqualError(t, err, "encryption must be NONE or ENCRYPTED")
}
func Test_Validate_WhenStoragePeriodIsEmpty_ValidationFails(t *testing.T) {
func Test_Validate_WhenRetentionTimePeriodIsEmpty_ValidationFails(t *testing.T) {
config := createValidBackupConfig()
config.StorePeriod = ""
config.RetentionTimePeriod = ""
plan := createUnlimitedPlan()
err := config.Validate(plan)
assert.EqualError(t, err, "store period is required")
assert.EqualError(t, err, "retention time period is required")
}
func Test_Validate_WhenMaxBackupSizeIsNegative_ValidationFails(t *testing.T) {
@@ -282,8 +284,8 @@ func Test_Validate_WhenMaxTotalSizeIsNegative_ValidationFails(t *testing.T) {
func Test_Validate_WhenPlanLimitsAreAtBoundary_ValidationWorks(t *testing.T) {
tests := []struct {
name string
configPeriod period.Period
planPeriod period.Period
configPeriod period.TimePeriod
planPeriod period.TimePeriod
configSize int64
planSize int64
configTotal int64
@@ -345,7 +347,7 @@ func Test_Validate_WhenPlanLimitsAreAtBoundary_ValidationWorks(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
config := createValidBackupConfig()
config.StorePeriod = tt.configPeriod
config.RetentionTimePeriod = tt.configPeriod
config.MaxBackupSizeMB = tt.configSize
config.MaxBackupsTotalSizeMB = tt.configTotal
@@ -364,12 +366,96 @@ func Test_Validate_WhenPlanLimitsAreAtBoundary_ValidationWorks(t *testing.T) {
}
}
func Test_Validate_WhenPolicyTypeIsCount_RequiresPositiveCount(t *testing.T) {
config := createValidBackupConfig()
config.RetentionPolicyType = RetentionPolicyTypeCount
config.RetentionCount = 0
plan := createUnlimitedPlan()
err := config.Validate(plan)
assert.EqualError(t, err, "retention count must be greater than 0")
}
func Test_Validate_WhenPolicyTypeIsCount_WithPositiveCount_ValidationPasses(t *testing.T) {
config := createValidBackupConfig()
config.RetentionPolicyType = RetentionPolicyTypeCount
config.RetentionCount = 10
plan := createUnlimitedPlan()
err := config.Validate(plan)
assert.NoError(t, err)
}
func Test_Validate_WhenPolicyTypeIsGFS_RequiresAtLeastOneField(t *testing.T) {
config := createValidBackupConfig()
config.RetentionPolicyType = RetentionPolicyTypeGFS
config.RetentionGfsDays = 0
config.RetentionGfsWeeks = 0
config.RetentionGfsMonths = 0
config.RetentionGfsYears = 0
plan := createUnlimitedPlan()
err := config.Validate(plan)
assert.EqualError(t, err, "at least one GFS retention field must be greater than 0")
}
func Test_Validate_WhenPolicyTypeIsGFS_WithOnlyHours_ValidationPasses(t *testing.T) {
config := createValidBackupConfig()
config.RetentionPolicyType = RetentionPolicyTypeGFS
config.RetentionGfsHours = 24
plan := createUnlimitedPlan()
err := config.Validate(plan)
assert.NoError(t, err)
}
func Test_Validate_WhenPolicyTypeIsGFS_WithOnlyDays_ValidationPasses(t *testing.T) {
config := createValidBackupConfig()
config.RetentionPolicyType = RetentionPolicyTypeGFS
config.RetentionGfsDays = 7
plan := createUnlimitedPlan()
err := config.Validate(plan)
assert.NoError(t, err)
}
func Test_Validate_WhenPolicyTypeIsGFS_WithAllFields_ValidationPasses(t *testing.T) {
config := createValidBackupConfig()
config.RetentionPolicyType = RetentionPolicyTypeGFS
config.RetentionGfsHours = 24
config.RetentionGfsDays = 7
config.RetentionGfsWeeks = 4
config.RetentionGfsMonths = 12
config.RetentionGfsYears = 3
plan := createUnlimitedPlan()
err := config.Validate(plan)
assert.NoError(t, err)
}
func Test_Validate_WhenPolicyTypeIsInvalid_ValidationFails(t *testing.T) {
config := createValidBackupConfig()
config.RetentionPolicyType = "INVALID"
plan := createUnlimitedPlan()
err := config.Validate(plan)
assert.EqualError(t, err, "invalid retention policy type")
}
func createValidBackupConfig() *BackupConfig {
intervalID := uuid.New()
return &BackupConfig{
DatabaseID: uuid.New(),
IsBackupsEnabled: true,
StorePeriod: period.PeriodMonth,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodMonth,
BackupIntervalID: intervalID,
BackupInterval: &intervals.Interval{ID: intervalID},
SendNotificationsOn: []BackupNotificationType{},

View File

@@ -227,7 +227,8 @@ func (s *BackupConfigService) initializeDefaultConfig(
_, err = s.backupConfigRepository.Save(&BackupConfig{
DatabaseID: databaseID,
IsBackupsEnabled: false,
StorePeriod: plan.MaxStoragePeriod,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: plan.MaxStoragePeriod,
MaxBackupSizeMB: plan.MaxBackupSizeMB,
MaxBackupsTotalSizeMB: plan.MaxBackupsTotalSizeMB,
BackupInterval: &intervals.Interval{

View File

@@ -35,9 +35,10 @@ func Test_AttachStorageFromSameWorkspace_SuccessfullyAttached(t *testing.T) {
timeOfDay := "04:00"
request := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -87,9 +88,10 @@ func Test_AttachStorageFromDifferentWorkspace_ReturnsForbidden(t *testing.T) {
timeOfDay := "04:00"
request := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -131,9 +133,10 @@ func Test_DeleteStorageWithAttachedDatabases_CannotDelete(t *testing.T) {
timeOfDay := "04:00"
request := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,
@@ -191,9 +194,10 @@ func Test_TransferStorageWithAttachedDatabase_CannotTransfer(t *testing.T) {
timeOfDay := "04:00"
request := BackupConfig{
DatabaseID: database.ID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodWeek,
DatabaseID: database.ID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodWeek,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,

View File

@@ -15,9 +15,10 @@ func EnableBackupsForTestDatabase(
timeOfDay := "16:00"
backupConfig := &BackupConfig{
DatabaseID: databaseID,
IsBackupsEnabled: true,
StorePeriod: period.PeriodDay,
DatabaseID: databaseID,
IsBackupsEnabled: true,
RetentionPolicyType: RetentionPolicyTypeTimePeriod,
RetentionTimePeriod: period.PeriodDay,
BackupInterval: &intervals.Interval{
Interval: intervals.IntervalDaily,
TimeOfDay: &timeOfDay,

View File

@@ -25,15 +25,16 @@ type MongodbDatabase struct {
Version tools.MongodbVersion `json:"version" gorm:"type:text;not null"`
Host string `json:"host" gorm:"type:text;not null"`
Port *int `json:"port" gorm:"type:int"`
Username string `json:"username" gorm:"type:text;not null"`
Password string `json:"password" gorm:"type:text;not null"`
Database string `json:"database" gorm:"type:text;not null"`
AuthDatabase string `json:"authDatabase" gorm:"type:text;not null;default:'admin'"`
IsHttps bool `json:"isHttps" gorm:"type:boolean;default:false"`
IsSrv bool `json:"isSrv" gorm:"column:is_srv;type:boolean;not null;default:false"`
CpuCount int `json:"cpuCount" gorm:"column:cpu_count;type:int;not null;default:1"`
Host string `json:"host" gorm:"type:text;not null"`
Port *int `json:"port" gorm:"type:int"`
Username string `json:"username" gorm:"type:text;not null"`
Password string `json:"password" gorm:"type:text;not null"`
Database string `json:"database" gorm:"type:text;not null"`
AuthDatabase string `json:"authDatabase" gorm:"type:text;not null;default:'admin'"`
IsHttps bool `json:"isHttps" gorm:"type:boolean;default:false"`
IsSrv bool `json:"isSrv" gorm:"column:is_srv;type:boolean;not null;default:false"`
IsDirectConnection bool `json:"isDirectConnection" gorm:"column:is_direct_connection;type:boolean;not null;default:false"`
CpuCount int `json:"cpuCount" gorm:"column:cpu_count;type:int;not null;default:1"`
}
func (m *MongodbDatabase) TableName() string {
@@ -132,6 +133,7 @@ func (m *MongodbDatabase) Update(incoming *MongodbDatabase) {
m.AuthDatabase = incoming.AuthDatabase
m.IsHttps = incoming.IsHttps
m.IsSrv = incoming.IsSrv
m.IsDirectConnection = incoming.IsDirectConnection
m.CpuCount = incoming.CpuCount
if incoming.Password != "" {
@@ -457,9 +459,12 @@ func (m *MongodbDatabase) buildConnectionURI(password string) string {
authDB = "admin"
}
tlsParams := ""
extraParams := ""
if m.IsHttps {
tlsParams = "&tls=true&tlsInsecure=true"
extraParams += "&tls=true&tlsInsecure=true"
}
if m.IsDirectConnection {
extraParams += "&directConnection=true"
}
if m.IsSrv {
@@ -470,7 +475,7 @@ func (m *MongodbDatabase) buildConnectionURI(password string) string {
m.Host,
m.Database,
authDB,
tlsParams,
extraParams,
)
}
@@ -487,7 +492,7 @@ func (m *MongodbDatabase) buildConnectionURI(password string) string {
port,
m.Database,
authDB,
tlsParams,
extraParams,
)
}
@@ -498,9 +503,12 @@ func (m *MongodbDatabase) BuildMongodumpURI(password string) string {
authDB = "admin"
}
tlsParams := ""
extraParams := ""
if m.IsHttps {
tlsParams = "&tls=true&tlsInsecure=true"
extraParams += "&tls=true&tlsInsecure=true"
}
if m.IsDirectConnection {
extraParams += "&directConnection=true"
}
if m.IsSrv {
@@ -510,7 +518,7 @@ func (m *MongodbDatabase) BuildMongodumpURI(password string) string {
url.QueryEscape(password),
m.Host,
authDB,
tlsParams,
extraParams,
)
}
@@ -526,7 +534,7 @@ func (m *MongodbDatabase) BuildMongodumpURI(password string) string {
m.Host,
port,
authDB,
tlsParams,
extraParams,
)
}

View File

@@ -631,6 +631,89 @@ func Test_Validate_SrvConnection_AllowsNullPort(t *testing.T) {
assert.NoError(t, err)
}
func Test_BuildConnectionURI_WithDirectConnection_ReturnsCorrectUri(t *testing.T) {
port := 27017
model := &MongodbDatabase{
Host: "mongo.example.local",
Port: &port,
Username: "testuser",
Password: "testpass123",
Database: "mydb",
AuthDatabase: "admin",
IsHttps: false,
IsSrv: false,
IsDirectConnection: true,
}
uri := model.buildConnectionURI("testpass123")
assert.Contains(t, uri, "mongodb://")
assert.Contains(t, uri, "directConnection=true")
assert.Contains(t, uri, "mongo.example.local:27017")
assert.Contains(t, uri, "authSource=admin")
}
func Test_BuildConnectionURI_WithoutDirectConnection_OmitsParam(t *testing.T) {
port := 27017
model := &MongodbDatabase{
Host: "localhost",
Port: &port,
Username: "testuser",
Password: "testpass123",
Database: "mydb",
AuthDatabase: "admin",
IsHttps: false,
IsSrv: false,
IsDirectConnection: false,
}
uri := model.buildConnectionURI("testpass123")
assert.NotContains(t, uri, "directConnection")
}
func Test_BuildMongodumpURI_WithDirectConnection_ReturnsCorrectUri(t *testing.T) {
port := 27017
model := &MongodbDatabase{
Host: "mongo.example.local",
Port: &port,
Username: "testuser",
Password: "testpass123",
Database: "mydb",
AuthDatabase: "admin",
IsHttps: false,
IsSrv: false,
IsDirectConnection: true,
}
uri := model.BuildMongodumpURI("testpass123")
assert.Contains(t, uri, "mongodb://")
assert.Contains(t, uri, "directConnection=true")
assert.NotContains(t, uri, "/mydb")
}
func Test_BuildConnectionURI_WithDirectConnectionAndTls_ReturnsBothParams(t *testing.T) {
port := 27017
model := &MongodbDatabase{
Host: "mongo.example.local",
Port: &port,
Username: "testuser",
Password: "testpass123",
Database: "mydb",
AuthDatabase: "admin",
IsHttps: true,
IsSrv: false,
IsDirectConnection: true,
}
uri := model.buildConnectionURI("testpass123")
assert.Contains(t, uri, "directConnection=true")
assert.Contains(t, uri, "tls=true")
assert.Contains(t, uri, "tlsInsecure=true")
}
func Test_Validate_StandardConnection_RequiresPort(t *testing.T) {
model := &MongodbDatabase{
Host: "localhost",

View File

@@ -14,6 +14,7 @@ var emailSMTPSender = &EmailSMTPSender{
env.SMTPPort,
env.SMTPUser,
env.SMTPPassword,
env.SMTPFrom,
env.SMTPHost != "" && env.SMTPPort != 0,
}

View File

@@ -24,6 +24,7 @@ type EmailSMTPSender struct {
smtpPort int
smtpUser string
smtpPassword string
smtpFrom string
isConfigured bool
}
@@ -33,7 +34,10 @@ func (s *EmailSMTPSender) SendEmail(to, subject, body string) error {
return nil
}
from := s.smtpUser
from := s.smtpFrom
if from == "" {
from = s.smtpUser
}
if from == "" {
from = "noreply@" + s.smtpHost
}

View File

@@ -9,17 +9,17 @@ import (
"mime"
"net"
"net/smtp"
"os"
"time"
"github.com/google/uuid"
)
const (
ImplicitTLSPort = 465
DefaultTimeout = 5 * time.Second
DefaultHelloName = "localhost"
MIMETypeHTML = "text/html"
MIMECharsetUTF8 = "UTF-8"
ImplicitTLSPort = 465
DefaultTimeout = 5 * time.Second
MIMETypeHTML = "text/html"
MIMECharsetUTF8 = "UTF-8"
)
type EmailNotifier struct {
@@ -116,6 +116,16 @@ func (e *EmailNotifier) EncryptSensitiveData(encryptor encryption.FieldEncryptor
return nil
}
func getHelloName() string {
hostname, err := os.Hostname()
if err != nil || hostname == "" {
return "localhost"
}
return hostname
}
// encodeRFC2047 encodes a string using RFC 2047 MIME encoding for email headers
// This ensures compatibility with SMTP servers that don't support SMTPUTF8
func encodeRFC2047(s string) string {
@@ -131,6 +141,7 @@ func (e *EmailNotifier) buildEmailContent(heading, message, from string) []byte
encodedSubject := encodeRFC2047(heading)
subject := fmt.Sprintf("Subject: %s\r\n", encodedSubject)
dateHeader := fmt.Sprintf("Date: %s\r\n", time.Now().UTC().Format(time.RFC1123Z))
messageID := fmt.Sprintf("Message-ID: <%s@%s>\r\n", uuid.New().String(), e.SMTPHost)
mimeHeaders := fmt.Sprintf(
"MIME-version: 1.0;\nContent-Type: %s; charset=\"%s\";\n\n",
@@ -144,7 +155,7 @@ func (e *EmailNotifier) buildEmailContent(heading, message, from string) []byte
toHeader := fmt.Sprintf("To: %s\r\n", e.TargetEmail)
return []byte(fromHeader + toHeader + subject + dateHeader + mimeHeaders + message)
return []byte(fromHeader + toHeader + subject + dateHeader + messageID + mimeHeaders + message)
}
func (e *EmailNotifier) sendImplicitTLS(
@@ -219,7 +230,7 @@ func (e *EmailNotifier) createStartTLSClient() (*smtp.Client, func(), error) {
return nil, nil, fmt.Errorf("failed to create SMTP client: %w", err)
}
if err := client.Hello(DefaultHelloName); err != nil {
if err := client.Hello(getHelloName()); err != nil {
_ = client.Quit()
_ = conn.Close()
return nil, nil, fmt.Errorf("SMTP hello failed: %w", err)

View File

@@ -9,9 +9,9 @@ import (
type DatabasePlan struct {
DatabaseID uuid.UUID `json:"databaseId" gorm:"column:database_id;type:uuid;primaryKey;not null"`
MaxBackupSizeMB int64 `json:"maxBackupSizeMb" gorm:"column:max_backup_size_mb;type:int;not null"`
MaxBackupsTotalSizeMB int64 `json:"maxBackupsTotalSizeMb" gorm:"column:max_backups_total_size_mb;type:int;not null"`
MaxStoragePeriod period.Period `json:"maxStoragePeriod" gorm:"column:max_storage_period;type:text;not null"`
MaxBackupSizeMB int64 `json:"maxBackupSizeMb" gorm:"column:max_backup_size_mb;type:int;not null"`
MaxBackupsTotalSizeMB int64 `json:"maxBackupsTotalSizeMb" gorm:"column:max_backups_total_size_mb;type:int;not null"`
MaxStoragePeriod period.TimePeriod `json:"maxStoragePeriod" gorm:"column:max_storage_period;type:text;not null"`
}
func (p *DatabasePlan) TableName() string {

View File

@@ -2,24 +2,24 @@ package period
import "time"
type Period string
type TimePeriod string
const (
PeriodDay Period = "DAY"
PeriodWeek Period = "WEEK"
PeriodMonth Period = "MONTH"
Period3Month Period = "3_MONTH"
Period6Month Period = "6_MONTH"
PeriodYear Period = "YEAR"
Period2Years Period = "2_YEARS"
Period3Years Period = "3_YEARS"
Period4Years Period = "4_YEARS"
Period5Years Period = "5_YEARS"
PeriodForever Period = "FOREVER"
PeriodDay TimePeriod = "DAY"
PeriodWeek TimePeriod = "WEEK"
PeriodMonth TimePeriod = "MONTH"
Period3Month TimePeriod = "3_MONTH"
Period6Month TimePeriod = "6_MONTH"
PeriodYear TimePeriod = "YEAR"
Period2Years TimePeriod = "2_YEARS"
Period3Years TimePeriod = "3_YEARS"
Period4Years TimePeriod = "4_YEARS"
Period5Years TimePeriod = "5_YEARS"
PeriodForever TimePeriod = "FOREVER"
)
// ToDuration converts Period to time.Duration
func (p Period) ToDuration() time.Duration {
func (p TimePeriod) ToDuration() time.Duration {
switch p {
case PeriodDay:
return 24 * time.Hour
@@ -55,7 +55,7 @@ func (p Period) ToDuration() time.Duration {
// 1 if p > other
//
// FOREVER is treated as the longest period
func (p Period) CompareTo(other Period) int {
func (p TimePeriod) CompareTo(other TimePeriod) int {
if p == other {
return 0
}

View File

@@ -0,0 +1,38 @@
-- +goose Up
ALTER TABLE backup_configs
ADD COLUMN retention_policy_type TEXT NOT NULL DEFAULT 'TIME_PERIOD',
ADD COLUMN retention_time_period TEXT NOT NULL DEFAULT '',
ADD COLUMN retention_count INT NOT NULL DEFAULT 0,
ADD COLUMN retention_gfs_hours INT NOT NULL DEFAULT 0,
ADD COLUMN retention_gfs_days INT NOT NULL DEFAULT 0,
ADD COLUMN retention_gfs_weeks INT NOT NULL DEFAULT 0,
ADD COLUMN retention_gfs_months INT NOT NULL DEFAULT 0,
ADD COLUMN retention_gfs_years INT NOT NULL DEFAULT 0;
UPDATE backup_configs
SET retention_time_period = store_period;
ALTER TABLE backup_configs
DROP COLUMN store_period;
-- +goose Down
ALTER TABLE backup_configs
ADD COLUMN store_period TEXT NOT NULL DEFAULT 'WEEK';
UPDATE backup_configs
SET store_period = CASE
WHEN retention_time_period != '' THEN retention_time_period
ELSE 'WEEK'
END;
ALTER TABLE backup_configs
DROP COLUMN retention_policy_type,
DROP COLUMN retention_time_period,
DROP COLUMN retention_count,
DROP COLUMN retention_gfs_hours,
DROP COLUMN retention_gfs_days,
DROP COLUMN retention_gfs_weeks,
DROP COLUMN retention_gfs_months,
DROP COLUMN retention_gfs_years;

View File

@@ -0,0 +1,9 @@
-- +goose Up
-- +goose StatementBegin
ALTER TABLE mongodb_databases ADD COLUMN is_direct_connection BOOLEAN NOT NULL DEFAULT FALSE;
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
ALTER TABLE mongodb_databases DROP COLUMN is_direct_connection;
-- +goose StatementEnd

View File

@@ -5,5 +5,6 @@ export type { Backup } from './model/Backup';
export type { BackupConfig } from './model/BackupConfig';
export { BackupNotificationType } from './model/BackupNotificationType';
export { BackupEncryption } from './model/BackupEncryption';
export { RetentionPolicyType } from './model/RetentionPolicyType';
export type { TransferDatabaseRequest } from './model/TransferDatabaseRequest';
export type { DatabasePlan } from '../plan';

View File

@@ -3,12 +3,22 @@ import type { Interval } from '../../intervals';
import type { Storage } from '../../storages';
import { BackupEncryption } from './BackupEncryption';
import type { BackupNotificationType } from './BackupNotificationType';
import type { RetentionPolicyType } from './RetentionPolicyType';
export interface BackupConfig {
databaseId: string;
isBackupsEnabled: boolean;
storePeriod: Period;
retentionPolicyType: RetentionPolicyType;
retentionTimePeriod: Period;
retentionCount: number;
retentionGfsHours: number;
retentionGfsDays: number;
retentionGfsWeeks: number;
retentionGfsMonths: number;
retentionGfsYears: number;
backupInterval?: Interval;
storage?: Storage;
sendNotificationsOn: BackupNotificationType[];

View File

@@ -0,0 +1,5 @@
export enum RetentionPolicyType {
TimePeriod = 'TIME_PERIOD',
Count = 'COUNT',
GFS = 'GFS',
}

View File

@@ -456,6 +456,46 @@ describe('MongodbConnectionStringParser', () => {
});
});
describe('Direct Connection Handling', () => {
it('should parse directConnection=true from URI', () => {
const result = expectSuccess(
MongodbConnectionStringParser.parse(
'mongodb://user:pass@host:27017/db?authSource=admin&directConnection=true',
),
);
expect(result.isDirectConnection).toBe(true);
});
it('should default isDirectConnection to false when not specified in URI', () => {
const result = expectSuccess(
MongodbConnectionStringParser.parse('mongodb://user:pass@host:27017/db'),
);
expect(result.isDirectConnection).toBe(false);
});
it('should parse isDirectConnection=true from key-value format', () => {
const result = expectSuccess(
MongodbConnectionStringParser.parse(
'host=localhost port=27017 database=mydb user=admin password=secret directConnection=true',
),
);
expect(result.isDirectConnection).toBe(true);
});
it('should default isDirectConnection to false in key-value format when not specified', () => {
const result = expectSuccess(
MongodbConnectionStringParser.parse(
'host=localhost port=27017 database=mydb user=admin password=secret',
),
);
expect(result.isDirectConnection).toBe(false);
});
});
describe('Password Placeholder Handling', () => {
it('should treat <db_password> placeholder as empty password in URI format', () => {
const result = expectSuccess(

View File

@@ -7,6 +7,7 @@ export type ParseResult = {
authDatabase: string;
useTls: boolean;
isSrv: boolean;
isDirectConnection: boolean;
};
export type ParseError = {
@@ -69,6 +70,7 @@ export class MongodbConnectionStringParser {
const database = decodeURIComponent(url.pathname.slice(1));
const authDatabase = this.getAuthSource(url.search) || 'admin';
const useTls = isSrv ? true : this.checkTlsMode(url.search);
const isDirectConnection = this.checkDirectConnection(url.search);
if (!host) {
return { error: 'Host is missing from connection string' };
@@ -87,6 +89,7 @@ export class MongodbConnectionStringParser {
authDatabase,
useTls,
isSrv,
isDirectConnection,
};
} catch (e) {
return {
@@ -133,6 +136,7 @@ export class MongodbConnectionStringParser {
}
const useTls = this.isTlsEnabled(tls);
const isDirectConnection = params['directConnection'] === 'true';
return {
host,
@@ -143,6 +147,7 @@ export class MongodbConnectionStringParser {
authDatabase,
useTls,
isSrv: false,
isDirectConnection,
};
} catch (e) {
return {
@@ -162,6 +167,16 @@ export class MongodbConnectionStringParser {
return params.get('authSource') || params.get('authDatabase') || undefined;
}
private static checkDirectConnection(queryString: string | undefined | null): boolean {
if (!queryString) return false;
const params = new URLSearchParams(
queryString.startsWith('?') ? queryString.slice(1) : queryString,
);
return params.get('directConnection') === 'true';
}
private static checkTlsMode(queryString: string | undefined | null): boolean {
if (!queryString) return false;

View File

@@ -11,5 +11,6 @@ export interface MongodbDatabase {
authDatabase: string;
isHttps: boolean;
isSrv: boolean;
isDirectConnection: boolean;
cpuCount: number;
}

View File

@@ -56,8 +56,8 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
const [showingRestoresBackupId, setShowingRestoresBackupId] = useState<string | undefined>();
const isReloadInProgress = useRef(false);
const isLazyLoadInProgress = useRef(false);
const lastRequestTimeRef = useRef<number>(0);
const isBackupsRequestInFlightRef = useRef(false);
const [downloadingBackupId, setDownloadingBackupId] = useState<string | undefined>();
const [cancellingBackupId, setCancellingBackupId] = useState<string | undefined>();
@@ -73,85 +73,59 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
};
const loadBackups = async (limit?: number) => {
if (isReloadInProgress.current || isLazyLoadInProgress.current) {
return;
}
if (isBackupsRequestInFlightRef.current) return;
isBackupsRequestInFlightRef.current = true;
isReloadInProgress.current = true;
const requestTime = Date.now();
lastRequestTimeRef.current = requestTime;
const loadLimit = limit ?? currentLimit;
try {
const loadLimit = limit || currentLimit;
const response = await backupsApi.getBackups(database.id, loadLimit, 0);
if (lastRequestTimeRef.current !== requestTime) return;
setBackups(response.backups);
setTotalBackups(response.total);
setHasMore(response.backups.length < response.total);
} catch (e) {
alert((e as Error).message);
if (lastRequestTimeRef.current === requestTime) {
alert((e as Error).message);
}
} finally {
isBackupsRequestInFlightRef.current = false;
}
isReloadInProgress.current = false;
};
const reloadInProgressBackups = async () => {
if (isReloadInProgress.current || isLazyLoadInProgress.current) {
return;
}
isReloadInProgress.current = true;
try {
// Fetch only the recent backups that could be in progress
// We fetch a small number (20) to capture recent backups that might be in progress
const response = await backupsApi.getBackups(database.id, 20, 0);
// Update only the backups that exist in both lists
setBackups((prevBackups) => {
const updatedBackups = [...prevBackups];
response.backups.forEach((newBackup) => {
const index = updatedBackups.findIndex((b) => b.id === newBackup.id);
if (index !== -1) {
updatedBackups[index] = newBackup;
} else if (index === -1 && updatedBackups.length < currentLimit) {
// New backup that doesn't exist yet (e.g., just created)
updatedBackups.unshift(newBackup);
}
});
return updatedBackups;
});
setTotalBackups(response.total);
} catch (e) {
alert((e as Error).message);
}
isReloadInProgress.current = false;
};
const loadMoreBackups = async () => {
if (isLoadingMore || !hasMore || isLazyLoadInProgress.current) {
if (isLoadingMore || !hasMore) {
return;
}
isLazyLoadInProgress.current = true;
setIsLoadingMore(true);
const newLimit = currentLimit + BACKUPS_PAGE_SIZE;
setCurrentLimit(newLimit);
const requestTime = Date.now();
lastRequestTimeRef.current = requestTime;
try {
const newLimit = currentLimit + BACKUPS_PAGE_SIZE;
const response = await backupsApi.getBackups(database.id, newLimit, 0);
if (lastRequestTimeRef.current !== requestTime) return;
setBackups(response.backups);
setCurrentLimit(newLimit);
setTotalBackups(response.total);
setHasMore(response.backups.length < response.total);
} catch (e) {
alert((e as Error).message);
if (lastRequestTimeRef.current === requestTime) {
alert((e as Error).message);
}
}
setIsLoadingMore(false);
isLazyLoadInProgress.current = false;
};
const makeBackup = async () => {
@@ -196,7 +170,7 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
try {
await backupsApi.cancelBackup(backupId);
await reloadInProgressBackups();
await loadBackups();
} catch (e) {
alert((e as Error).message);
}
@@ -220,22 +194,13 @@ export const BackupsComponent = ({ database, isCanManageDBs, scrollContainerRef
return () => {};
}, [database]);
// Reload backups that are in progress to update their state
useEffect(() => {
const hasInProgressBackups = backups.some(
(backup) => backup.status === BackupStatus.IN_PROGRESS,
);
if (!hasInProgressBackups) {
return;
}
const timeoutId = setTimeout(async () => {
await reloadInProgressBackups();
const intervalId = setInterval(() => {
loadBackups();
}, 1_000);
return () => clearTimeout(timeoutId);
}, [backups]);
return () => clearInterval(intervalId);
}, [currentLimit]);
useEffect(() => {
if (downloadingBackupId) {

View File

@@ -20,6 +20,7 @@ import {
type BackupConfig,
BackupEncryption,
type DatabasePlan,
RetentionPolicyType,
backupConfigApi,
} from '../../../entity/backups';
import { BackupNotificationType } from '../../../entity/backups/model/BackupNotificationType';
@@ -64,6 +65,15 @@ const weekdayOptions = [
{ value: 7, label: 'Sun' },
];
const retentionPolicyOptions = [
{
label: 'GFS (keep last N hourly, daily, weekly, monthly and yearly backups)',
value: RetentionPolicyType.GFS,
},
{ label: 'Time period (last N days)', value: RetentionPolicyType.TimePeriod },
{ label: 'Count (N last backups)', value: RetentionPolicyType.Count },
];
export const EditBackupConfigComponent = ({
user,
database,
@@ -95,6 +105,7 @@ export const EditBackupConfigComponent = ({
(backupConfig?.maxBackupSizeMb ?? 0) > 0 ||
(backupConfig?.maxBackupsTotalSizeMb ?? 0) > 0;
const [isShowAdvanced, setShowAdvanced] = useState(hasAdvancedValues);
const [isShowGfsHint, setShowGfsHint] = useState(false);
const timeFormat = useMemo(() => {
const is12 = getIs12Hour();
@@ -242,8 +253,20 @@ export const EditBackupConfigComponent = ({
timeOfDay: '00:00',
},
storage: undefined,
storePeriod:
plan.maxStoragePeriod === Period.FOREVER ? Period.THREE_MONTH : plan.maxStoragePeriod,
retentionPolicyType: IS_CLOUD
? RetentionPolicyType.GFS
: RetentionPolicyType.TimePeriod,
retentionTimePeriod: IS_CLOUD
? plan.maxStoragePeriod === Period.FOREVER
? Period.THREE_MONTH
: plan.maxStoragePeriod
: Period.THREE_MONTH,
retentionCount: 100,
retentionGfsHours: 24,
retentionGfsDays: 7,
retentionGfsWeeks: 4,
retentionGfsMonths: 12,
retentionGfsYears: 3,
sendNotificationsOn: [BackupNotificationType.BackupFailed],
isRetryIfFailed: true,
maxFailedTriesCount: 3,
@@ -295,10 +318,27 @@ export const EditBackupConfigComponent = ({
? getLocalDayOfMonth(backupInterval.dayOfMonth, backupInterval.timeOfDay)
: backupInterval?.dayOfMonth;
// mandatory-field check
const retentionPolicyType = backupConfig.retentionPolicyType ?? RetentionPolicyType.TimePeriod;
const isRetentionValid = (() => {
switch (retentionPolicyType) {
case RetentionPolicyType.TimePeriod:
return Boolean(backupConfig.retentionTimePeriod);
case RetentionPolicyType.Count:
return (backupConfig.retentionCount ?? 0) > 0;
case RetentionPolicyType.GFS:
return (
(backupConfig.retentionGfsDays ?? 0) > 0 ||
(backupConfig.retentionGfsWeeks ?? 0) > 0 ||
(backupConfig.retentionGfsMonths ?? 0) > 0 ||
(backupConfig.retentionGfsYears ?? 0) > 0
);
}
})();
const isAllFieldsFilled =
!backupConfig.isBackupsEnabled ||
(Boolean(backupConfig.storePeriod) &&
(isRetentionValid &&
Boolean(backupConfig.storage?.id) &&
Boolean(backupConfig.encryption) &&
Boolean(backupInterval?.interval) &&
@@ -467,7 +507,7 @@ export const EditBackupConfigComponent = ({
</>
)}
<div className="mt-2 mb-1 flex w-full flex-col items-start sm:flex-row sm:items-center">
<div className="mt-5 mb-1 flex w-full flex-col items-start sm:flex-row sm:items-center">
<div className="mb-1 min-w-[150px] sm:mb-0">Storage</div>
<div className="flex w-full items-center">
<Select
@@ -530,23 +570,160 @@ export const EditBackupConfigComponent = ({
</div>
)}
<div className="mb-1 flex w-full flex-col items-start sm:flex-row sm:items-center">
<div className="mb-1 min-w-[150px] sm:mb-0">Store period</div>
<div className="flex items-center">
<div className="mt-5 mb-1 flex w-full flex-col items-start sm:flex-row sm:items-start">
<div className="mt-1 mb-1 min-w-[150px] sm:mb-0">Retention policy</div>
<div className="flex flex-col gap-1">
<Select
value={backupConfig.storePeriod}
onChange={(v) => updateBackupConfig({ storePeriod: v })}
value={retentionPolicyType}
options={retentionPolicyOptions}
size="small"
className="w-[200px]"
options={availablePeriods}
popupMatchSelectWidth={false}
onChange={(v) => {
const type = v as RetentionPolicyType;
const updates: Partial<typeof backupConfig> = { retentionPolicyType: type };
if (type === RetentionPolicyType.GFS) {
updates.retentionGfsHours = 24;
updates.retentionGfsDays = 7;
updates.retentionGfsWeeks = 4;
updates.retentionGfsMonths = 12;
updates.retentionGfsYears = 3;
} else if (type === RetentionPolicyType.Count) {
updates.retentionCount = 100;
}
updateBackupConfig(updates);
}}
/>
<Tooltip
className="cursor-pointer"
title="How long to keep the backups? Make sure you have enough storage space."
>
<InfoCircleOutlined className="ml-2" style={{ color: 'gray' }} />
</Tooltip>
{retentionPolicyType === RetentionPolicyType.TimePeriod && (
<div className="flex items-center">
<Select
value={backupConfig.retentionTimePeriod}
onChange={(v) => updateBackupConfig({ retentionTimePeriod: v })}
size="small"
className="w-[200px]"
options={availablePeriods}
/>
<Tooltip
className="cursor-pointer"
title="How long to keep the backups. Backups older than this period are automatically deleted."
>
<InfoCircleOutlined className="ml-2" style={{ color: 'gray' }} />
</Tooltip>
</div>
)}
{retentionPolicyType === RetentionPolicyType.Count && (
<div className="flex items-center">
<InputNumber
min={1}
value={backupConfig.retentionCount}
onChange={(v) => updateBackupConfig({ retentionCount: v ?? 1 })}
size="small"
className="w-[80px]"
/>
<span className="ml-2 text-sm text-gray-600 dark:text-gray-400">
most recent backups
</span>
<Tooltip
className="cursor-pointer"
title="Keep only the specified number of most recent backups. Older backups beyond this count are automatically deleted."
>
<InfoCircleOutlined className="ml-2" style={{ color: 'gray' }} />
</Tooltip>
</div>
)}
{retentionPolicyType === RetentionPolicyType.GFS && (
<>
<div>
<span
className="cursor-pointer text-xs text-blue-600 hover:text-blue-800"
onClick={() => setShowGfsHint(!isShowGfsHint)}
>
{isShowGfsHint ? 'Hide' : 'What is GFS (Grandfather-Father-Son)?'}
</span>
{isShowGfsHint && (
<div className="mt-1 max-w-[280px] text-xs text-gray-600 dark:text-gray-400">
GFS (Grandfather-Father-Son) rotation: keep the last N hourly, daily, weekly,
monthly and yearly backups. This allows keeping backups over long periods of
time within a reasonable storage space.
</div>
)}
</div>
<div className="flex flex-col gap-1">
<div className="flex items-center gap-2">
<span className="w-[110px] text-sm text-gray-600 dark:text-gray-400">
Hourly backups
</span>
<InputNumber
min={0}
value={backupConfig.retentionGfsHours}
onChange={(v) => updateBackupConfig({ retentionGfsHours: v ?? 0 })}
size="small"
className="w-[80px]"
/>
</div>
<div className="flex items-center gap-2">
<span className="w-[110px] text-sm text-gray-600 dark:text-gray-400">
Daily backups
</span>
<InputNumber
min={0}
value={backupConfig.retentionGfsDays}
onChange={(v) => updateBackupConfig({ retentionGfsDays: v ?? 0 })}
size="small"
className="w-[80px]"
/>
</div>
<div className="flex items-center gap-2">
<span className="w-[110px] text-sm text-gray-600 dark:text-gray-400">
Weekly backups
</span>
<InputNumber
min={0}
value={backupConfig.retentionGfsWeeks}
onChange={(v) => updateBackupConfig({ retentionGfsWeeks: v ?? 0 })}
size="small"
className="w-[80px]"
/>
</div>
<div className="flex items-center gap-2">
<span className="w-[110px] text-sm text-gray-600 dark:text-gray-400">
Monthly backups
</span>
<InputNumber
min={0}
value={backupConfig.retentionGfsMonths}
onChange={(v) => updateBackupConfig({ retentionGfsMonths: v ?? 0 })}
size="small"
className="w-[80px]"
/>
</div>
<div className="flex items-center gap-2">
<span className="w-[110px] text-sm text-gray-600 dark:text-gray-400">
Yearly backups
</span>
<InputNumber
min={0}
value={backupConfig.retentionGfsYears}
onChange={(v) => updateBackupConfig({ retentionGfsYears: v ?? 0 })}
size="small"
className="w-[80px]"
/>
</div>
</div>
</>
)}
</div>
</div>

View File

@@ -6,7 +6,12 @@ import { useMemo } from 'react';
import { useEffect, useState } from 'react';
import { IS_CLOUD } from '../../../constants';
import { type BackupConfig, BackupEncryption, backupConfigApi } from '../../../entity/backups';
import {
type BackupConfig,
BackupEncryption,
RetentionPolicyType,
backupConfigApi,
} from '../../../entity/backups';
import { BackupNotificationType } from '../../../entity/backups/model/BackupNotificationType';
import type { Database } from '../../../entity/databases';
import { Period } from '../../../entity/databases/model/Period';
@@ -60,10 +65,21 @@ const notificationLabels = {
[BackupNotificationType.BackupSuccess]: 'Backup success',
};
const formatGfsRetention = (config: BackupConfig): string => {
const parts: string[] = [];
if (config.retentionGfsHours > 0) parts.push(`${config.retentionGfsHours} hourly`);
if (config.retentionGfsDays > 0) parts.push(`${config.retentionGfsDays} daily`);
if (config.retentionGfsWeeks > 0) parts.push(`${config.retentionGfsWeeks} weekly`);
if (config.retentionGfsMonths > 0) parts.push(`${config.retentionGfsMonths} monthly`);
if (config.retentionGfsYears > 0) parts.push(`${config.retentionGfsYears} yearly`);
return parts.length > 0 ? parts.join(', ') : 'Not configured';
};
export const ShowBackupConfigComponent = ({ database }: Props) => {
const [backupConfig, setBackupConfig] = useState<BackupConfig>();
// Detect user's preferred time format (12-hour vs 24-hour)
const timeFormat = useMemo(() => {
const is12Hour = getIs12Hour();
return {
@@ -92,7 +108,6 @@ export const ShowBackupConfigComponent = ({ database }: Props) => {
const formattedTime = localTime ? localTime.format(timeFormat.format) : '';
// Convert UTC weekday/day-of-month to local equivalents for display
const displayedWeekday: number | undefined =
backupInterval?.interval === IntervalType.WEEKLY &&
backupInterval.weekday &&
@@ -107,6 +122,8 @@ export const ShowBackupConfigComponent = ({ database }: Props) => {
? getLocalDayOfMonth(backupInterval.dayOfMonth, backupInterval.timeOfDay)
: backupInterval?.dayOfMonth;
const retentionPolicyType = backupConfig.retentionPolicyType ?? RetentionPolicyType.TimePeriod;
return (
<div>
<div className="mb-1 flex w-full items-center">
@@ -193,8 +210,27 @@ export const ShowBackupConfigComponent = ({ database }: Props) => {
)}
<div className="mb-1 flex w-full items-center">
<div className="min-w-[150px]">Store period</div>
<div>{backupConfig.storePeriod ? periodLabels[backupConfig.storePeriod] : ''}</div>
<div className="min-w-[150px]">Retention policy</div>
<div className="flex items-center gap-1">
{retentionPolicyType === RetentionPolicyType.TimePeriod && (
<span>
{backupConfig.retentionTimePeriod
? periodLabels[backupConfig.retentionTimePeriod]
: ''}
</span>
)}
{retentionPolicyType === RetentionPolicyType.Count && (
<span>Keep last {backupConfig.retentionCount} backups</span>
)}
{retentionPolicyType === RetentionPolicyType.GFS && (
<span className="flex items-center gap-1">
{formatGfsRetention(backupConfig)}
<Tooltip title="Grandfather-Father-Son rotation: keep the last N hourly, daily, weekly, monthly and yearly backups.">
<InfoCircleOutlined style={{ color: 'gray' }} />
</Tooltip>
</span>
)}
</div>
</div>
<div className="mb-1 flex w-full items-center">

View File

@@ -46,7 +46,10 @@ export const EditMongoDbSpecificDataComponent = ({
const [isTestingConnection, setIsTestingConnection] = useState(false);
const [isConnectionFailed, setIsConnectionFailed] = useState(false);
const hasAdvancedValues = !!database.mongodb?.authDatabase || !!database.mongodb?.isSrv;
const hasAdvancedValues =
!!database.mongodb?.authDatabase ||
!!database.mongodb?.isSrv ||
!!database.mongodb?.isDirectConnection;
const [isShowAdvanced, setShowAdvanced] = useState(hasAdvancedValues);
const parseFromClipboard = async () => {
@@ -80,11 +83,12 @@ export const EditMongoDbSpecificDataComponent = ({
authDatabase: result.authDatabase,
isHttps: result.useTls,
isSrv: result.isSrv,
isDirectConnection: result.isDirectConnection,
cpuCount: 1,
},
};
if (result.isSrv) {
if (result.isSrv || result.isDirectConnection) {
setShowAdvanced(true);
}
@@ -407,6 +411,31 @@ export const EditMongoDbSpecificDataComponent = ({
</div>
</div>
<div className="mb-1 flex w-full items-center">
<div className="min-w-[150px]">Direct connection</div>
<div className="flex items-center">
<Switch
checked={editingDatabase.mongodb?.isDirectConnection || false}
onChange={(checked) => {
if (!editingDatabase.mongodb) return;
setEditingDatabase({
...editingDatabase,
mongodb: { ...editingDatabase.mongodb, isDirectConnection: checked },
});
setIsConnectionTested(false);
}}
size="small"
/>
<Tooltip
className="cursor-pointer"
title="Connect directly to a single server, skipping replica set discovery. Useful when the server is behind a load balancer, proxy or tunnel."
>
<InfoCircleOutlined className="ml-2" style={{ color: 'gray' }} />
</Tooltip>
</div>
</div>
<div className="mb-1 flex w-full items-center">
<div className="min-w-[150px]">Auth database</div>
<Input

View File

@@ -42,6 +42,13 @@ export const ShowMongoDbSpecificDataComponent = ({ database }: Props) => {
<div>{database.mongodb?.cpuCount}</div>
</div>
{database.mongodb?.isDirectConnection && (
<div className="mb-1 flex w-full items-center">
<div className="min-w-[150px]">Direct connection</div>
<div>Yes</div>
</div>
)}
{database.mongodb?.authDatabase && (
<div className="mb-1 flex w-full items-center">
<div className="min-w-[150px]">Auth database</div>

View File

@@ -10,6 +10,9 @@ interface Props {
}
export function EditNASStorageComponent({ storage, setStorage, setUnsaved }: Props) {
const shareHasSlash =
storage?.nasStorage?.share?.includes('/') || storage?.nasStorage?.share?.includes('\\');
return (
<>
<div className="mb-1 flex w-full flex-col items-start sm:flex-row sm:items-center">
@@ -60,24 +63,33 @@ export function EditNASStorageComponent({ storage, setStorage, setUnsaved }: Pro
<div className="mb-1 flex w-full flex-col items-start sm:flex-row sm:items-center">
<div className="mb-1 min-w-[110px] sm:mb-0">Share</div>
<Input
value={storage?.nasStorage?.share || ''}
onChange={(e) => {
if (!storage?.nasStorage) return;
<div className="flex flex-col">
<Input
value={storage?.nasStorage?.share || ''}
onChange={(e) => {
if (!storage?.nasStorage) return;
setStorage({
...storage,
nasStorage: {
...storage.nasStorage,
share: e.target.value.trim(),
},
});
setUnsaved();
}}
size="small"
className="w-full max-w-[250px]"
placeholder="shared_folder"
/>
setStorage({
...storage,
nasStorage: {
...storage.nasStorage,
share: e.target.value.trim(),
},
});
setUnsaved();
}}
size="small"
className="w-full max-w-[250px]"
placeholder="shared_folder"
status={shareHasSlash ? 'warning' : undefined}
/>
{shareHasSlash && (
<div className="mt-1 max-w-[250px] text-xs text-yellow-600">
Share must be a single share name. Use the Path field for subdirectories (e.g. Share:
Databasus, Path: DB1)
</div>
)}
</div>
</div>
<div className="mb-1 flex w-full flex-col items-start sm:flex-row sm:items-center">