scaffolding

This commit is contained in:
Wayne
2025-07-10 13:32:54 +03:00
commit f243775ae6
31 changed files with 669 additions and 0 deletions

22
.env.example Normal file
View File

@@ -0,0 +1,22 @@
# Application
NODE_ENV=development
PORT_BACKEND=4000
PORT_FRONTEND=3000
# PostgreSQL
POSTGRES_DB=open_archive
POSTGRES_USER=admin
POSTGRES_PASSWORD=password
DATABASE_URL="postgresql://admin:password@postgres:5432/open_archive?schema=public"
# Redis
REDIS_HOST=redis
REDIS_PORT=6379
# Meilisearch
MEILI_MASTER_KEY=aSampleMasterKey
MEILI_HOST=http://meilisearch:7700
# JWT
JWT_SECRET="a-very-secret-key"
JWT_EXPIRES_IN="7d"

20
.gitignore vendored Normal file
View File

@@ -0,0 +1,20 @@
# Node
node_modules
dist
.env
.env.*
!/.env.example
# PNPM
pnpm-lock.yaml
pnpm-debug.log
# IDE
.vscode
.idea
# OS
.DS_Store
# Dev
.dev

121
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,121 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that are welcoming, open, and respectful.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
- Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
- The use of sexualized language or imagery, and sexual attention or
advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email
address, without their explicit permission
- Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
[INSERT CONTACT METHOD].
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interaction in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.1, available at
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html

49
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,49 @@
# Contributing to Open Archive
First off, thank you for considering contributing to Open Archive! It's people like you that make open source such a great community. We welcome any and all contributions.
## Where do I start?
Not sure where to start? You can:
- Look through the [open issues](https://github.com/your-username/open-archive/issues) for bugs or feature requests.
- Check the issues labeled `good first issue` for tasks that are a good entry point into the codebase.
## How to Contribute
### Reporting Bugs
If you find a bug, please ensure the bug was not already reported by searching on GitHub under [Issues](https://github.com/your-username/open-archive/issues). If you're unable to find an open issue addressing the problem, open a new one. Be sure to include a **title and clear description**, as much relevant information as possible, and a **code sample** or an **executable test case** demonstrating the expected behavior that is not occurring.
### Suggesting Enhancements
If you have an idea for an enhancement, please open an issue to discuss it. This allows us to coordinate our efforts and prevent duplication of work.
### Pull Requests
1. **Fork the repository** and create your branch from `main`.
2. **Set up your development environment** by following the instructions in the `README.md`.
3. **Make your changes.** Ensure your code follows the project's coding style.
4. **Write tests** for any new functionality.
5. **Ensure all tests pass** before submitting your pull request.
6. **Update the documentation** if your changes require it.
7. **Submit a pull request** to the `main` branch of the main repository. Please provide a clear description of the problem and solution. Include the relevant issue number if applicable.
## Code of Conduct
This project and everyone participating in it is governed by the [Open Archive Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior.
## Styleguides
### Git Commit Messages
- Use the present tense ("Add feature" not "Added feature").
- Use the imperative mood ("Move cursor to..." not "Moves cursor to...").
- Limit the first line to 72 characters or less.
- Reference issues and pull requests liberally after the first line.
### TypeScript Styleguide
- Follow the existing code style.
- Use TypeScript's strict mode.
- Avoid using `any` as a type. Define clear interfaces and types in the `packages/types` directory.

80
README.md Normal file
View File

@@ -0,0 +1,80 @@
# Open Archive
**A secure, sovereign, and affordable open-source platform for email archiving and eDiscovery.**
Open Archive empowers individuals and organizations to take control of their digital communication history. It provides a robust, self-hosted solution for ingesting, storing, and searching emails from major platforms, ensuring you have a permanent, tamper-proof record of your most critical data, free from vendor lock-in.
## Vision
To provide individuals and organizations with a secure, sovereign, and affordable platform to preserve and access their digital communication history, free from vendor lock-in.
## Key Features
- **Universal Ingestion**: Connect to Google Workspace, Microsoft 365, and standard IMAP servers to perform initial bulk imports and maintain continuous, real-time synchronization.
- **Secure & Efficient Storage**: Emails are stored in the standard `.eml` format. The system uses deduplication and compression to minimize storage costs. All data is encrypted at rest.
- **Pluggable Storage Backends**: Start with local filesystem storage and scale to S3-compatible object storage (like AWS S3 or MinIO) as your needs grow.
- **Powerful Search & eDiscovery**: A high-performance search engine indexes the full text of emails and attachments (PDF, DOCX, etc.). The intuitive UI supports advanced search operators, filtering, and case management.
- **Compliance & Retention**: Define granular retention policies to automatically manage the lifecycle of your data. Place legal holds on communications to prevent deletion during litigation.
- **Comprehensive Auditing**: An immutable audit trail logs all system activities, ensuring you have a clear record of who accessed what and when.
- **Role-Based Access Control (RBAC)**: Enforce the principle of least privilege with pre-defined roles for Administrators, Auditors, and End Users.
## Tech Stack
Open Archive is built on a modern, scalable, and maintainable technology stack:
- **Frontend**: SvelteKit with Svelte 5
- **Backend**: Node.js with Express.js & TypeScript
- **Job Queue**: BullMQ on Redis for robust, asynchronous processing
- **Search Engine**: Meilisearch for blazingly fast and resource-efficient search
- **Database**: PostgreSQL for metadata, user management, and audit logs
- **Deployment**: Docker Compose for easy, one-command deployment
## Getting Started
### Prerequisites
- [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/)
- A server or local machine with at least 2GB of RAM.
### Installation
1. **Clone the repository:**
```bash
git clone https://github.com/your-username/open-archive.git
cd open-archive
```
2. **Configure your environment:**
Copy the example environment file and customize it with your settings.
```bash
cp .env.example .env
```
You will need to edit the `.env` file to set your database passwords, secret keys, and other essential configuration.
3. **Run the application:**
```bash
docker-compose up -d
```
This command will build the necessary Docker images and start all the services (frontend, backend, database, etc.) in the background.
4. **Access the application:**
Once the services are running, you can access the Open Archive web interface by navigating to `http://localhost:5173` in your web browser.
## Contributing
We welcome contributions from the community! Whether you're a developer, a designer, or just an enthusiast, there are many ways to get involved.
- **Reporting Bugs**: If you find a bug, please open an issue on our GitHub repository.
- **Suggesting Enhancements**: Have an idea for a new feature? We'd love to hear it. Open an issue to start the discussion.
- **Code Contributions**: If you'd like to contribute code, please fork the repository and submit a pull request.
Please read our `CONTRIBUTING.md` file for more details on our code of conduct and the process for submitting pull requests.
## License
This project is licensed under the MIT License - see the `LICENSE` file for details.

66
docker-compose.yml Normal file
View File

@@ -0,0 +1,66 @@
version: '3.8'
services:
frontend:
build:
context: ./packages/frontend
dockerfile: Dockerfile
ports:
- '3000:3000'
depends_on:
- backend-api
env_file:
- ./.env
backend-api:
build:
context: ./packages/backend
dockerfile: Dockerfile
ports:
- '4000:4000'
depends_on:
- postgres
- redis
env_file:
- ./.env
worker:
build:
context: ./packages/backend
dockerfile: Dockerfile
command: 'pnpm run start:worker'
depends_on:
- postgres
- redis
env_file:
- ./.env
postgres:
image: postgres:15
ports:
- '5432:5432'
volumes:
- postgres_data:/var/lib/postgresql/data
env_file:
- ./.env
redis:
image: redis:7
ports:
- '6379:6379'
volumes:
- redis_data:/data
meilisearch:
image: getmeili/meilisearch:v1.3
ports:
- '7700:7700'
volumes:
- meili_data:/meili_data
env_file:
- ./.env
volumes:
postgres_data:
redis_data:
meili_data:

21
package.json Normal file
View File

@@ -0,0 +1,21 @@
{
"name": "open-archive",
"private": true,
"scripts": {
"dev": "pnpm --filter \"./packages/*\" --parallel dev",
"build": "pnpm --filter \"./packages/*\" --parallel build"
},
"devDependencies": {
"typescript": "^5.0.0"
},
"packageManager": "pnpm@10.13.1",
"engines": {
"node": ">=18.0.0",
"pnpm": ">=8.0.0"
},
"pnpm": {
"onlyBuiltDependencies": [
"esbuild"
]
}
}

View File

@@ -0,0 +1,24 @@
{
"name": "@open-archive/backend",
"version": "0.1.0",
"private": true,
"main": "dist/index.js",
"scripts": {
"dev": "ts-node-dev --respawn --transpile-only src/index.ts ",
"build": "tsc",
"start": "node dist/index.js",
"start:worker": "ts-node-dev --respawn --transpile-only src/workers/ingestion.worker.ts"
},
"dependencies": {
"@open-archive/types": "workspace:*",
"express": "^5.1.0",
"bullmq": "^5.56.3",
"dotenv": "^17.2.0"
},
"devDependencies": {
"@types/express": "^5.0.3",
"@types/node": "^24.0.12",
"ts-node-dev": "^2.0.0",
"typescript": "^5.8.3"
}
}

View File

@@ -0,0 +1,15 @@
import express from 'express';
import dotenv from 'dotenv';
dotenv.config();
const app = express();
const port = process.env.PORT_BACKEND || 8000;
app.get('/', (req, res) => {
res.send('Backend is running!');
});
app.listen(port, () => {
console.log(`Backend listening at http://localhost:${port}`);
});

View File

@@ -0,0 +1,10 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"composite": true
},
"include": ["src/**/*.ts"],
"exclude": ["node_modules", "dist"]
}

23
packages/frontend/.gitignore vendored Normal file
View File

@@ -0,0 +1,23 @@
node_modules
# Output
.output
.vercel
.netlify
.wrangler
/.svelte-kit
/build
# OS
.DS_Store
Thumbs.db
# Env
.env
.env.*
!.env.example
!.env.test
# Vite
vite.config.js.timestamp-*
vite.config.ts.timestamp-*

1
packages/frontend/.npmrc Normal file
View File

@@ -0,0 +1 @@
engine-strict=true

View File

@@ -0,0 +1,9 @@
# Package Managers
package-lock.json
pnpm-lock.yaml
yarn.lock
bun.lock
bun.lockb
# Miscellaneous
/static/

View File

@@ -0,0 +1,18 @@
{
"useTabs": true,
"singleQuote": true,
"trailingComma": "none",
"printWidth": 100,
"plugins": [
"prettier-plugin-svelte",
"prettier-plugin-tailwindcss"
],
"overrides": [
{
"files": "*.svelte",
"options": {
"parser": "svelte"
}
}
]
}

View File

@@ -0,0 +1,38 @@
# sv
Everything you need to build a Svelte project, powered by [`sv`](https://github.com/sveltejs/cli).
## Creating a project
If you're seeing this, you've probably already done this step. Congrats!
```bash
# create a new project in the current directory
npx sv create
# create a new project in my-app
npx sv create my-app
```
## Developing
Once you've created a project and installed dependencies with `npm install` (or `pnpm install` or `yarn`), start a development server:
```bash
npm run dev
# or start the server and open the app in a new browser tab
npm run dev -- --open
```
## Building
To create a production version of your app:
```bash
npm run build
```
You can preview the production build with `npm run preview`.
> To deploy your app, you may need to install an [adapter](https://svelte.dev/docs/kit/adapters) for your target environment.

View File

@@ -0,0 +1,30 @@
{
"name": "frontend",
"private": true,
"version": "0.0.1",
"type": "module",
"scripts": {
"dev": "vite dev",
"build": "vite build",
"preview": "vite preview",
"prepare": "svelte-kit sync || echo ''",
"check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json",
"check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch",
"format": "prettier --write .",
"lint": "prettier --check ."
},
"devDependencies": {
"@sveltejs/adapter-auto": "^6.0.0",
"@sveltejs/kit": "^2.16.0",
"@sveltejs/vite-plugin-svelte": "^5.0.0",
"@tailwindcss/vite": "^4.0.0",
"prettier": "^3.4.2",
"prettier-plugin-svelte": "^3.3.3",
"prettier-plugin-tailwindcss": "^0.6.11",
"svelte": "^5.0.0",
"svelte-check": "^4.0.0",
"tailwindcss": "^4.0.0",
"typescript": "^5.0.0",
"vite": "^6.2.6"
}
}

View File

@@ -0,0 +1 @@
@import 'tailwindcss';

13
packages/frontend/src/app.d.ts vendored Normal file
View File

@@ -0,0 +1,13 @@
// See https://svelte.dev/docs/kit/types#app.d.ts
// for information about these interfaces
declare global {
namespace App {
// interface Error {}
// interface Locals {}
// interface PageData {}
// interface PageState {}
// interface Platform {}
}
}
export {};

View File

@@ -0,0 +1,12 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<link rel="icon" href="%sveltekit.assets%/favicon.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
%sveltekit.head%
</head>
<body data-sveltekit-preload-data="hover">
<div style="display: contents">%sveltekit.body%</div>
</body>
</html>

View File

@@ -0,0 +1 @@
// place files you want to import through the `$lib` alias in this folder.

View File

@@ -0,0 +1,7 @@
<script lang="ts">
import '../app.css';
let { children } = $props();
</script>
{@render children()}

View File

@@ -0,0 +1,2 @@
<h1>Welcome to SvelteKit</h1>
<p>Visit <a href="https://svelte.dev/docs/kit">svelte.dev/docs/kit</a> to read the documentation</p>

View File

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="107" height="128" viewBox="0 0 107 128"><title>svelte-logo</title><path d="M94.157 22.819c-10.4-14.885-30.94-19.297-45.792-9.835L22.282 29.608A29.92 29.92 0 0 0 8.764 49.65a31.5 31.5 0 0 0 3.108 20.231 30 30 0 0 0-4.477 11.183 31.9 31.9 0 0 0 5.448 24.116c10.402 14.887 30.942 19.297 45.791 9.835l26.083-16.624A29.92 29.92 0 0 0 98.235 78.35a31.53 31.53 0 0 0-3.105-20.232 30 30 0 0 0 4.474-11.182 31.88 31.88 0 0 0-5.447-24.116" style="fill:#ff3e00"/><path d="M45.817 106.582a20.72 20.72 0 0 1-22.237-8.243 19.17 19.17 0 0 1-3.277-14.503 18 18 0 0 1 .624-2.435l.49-1.498 1.337.981a33.6 33.6 0 0 0 10.203 5.098l.97.294-.09.968a5.85 5.85 0 0 0 1.052 3.878 6.24 6.24 0 0 0 6.695 2.485 5.8 5.8 0 0 0 1.603-.704L69.27 76.28a5.43 5.43 0 0 0 2.45-3.631 5.8 5.8 0 0 0-.987-4.371 6.24 6.24 0 0 0-6.698-2.487 5.7 5.7 0 0 0-1.6.704l-9.953 6.345a19 19 0 0 1-5.296 2.326 20.72 20.72 0 0 1-22.237-8.243 19.17 19.17 0 0 1-3.277-14.502 17.99 17.99 0 0 1 8.13-12.052l26.081-16.623a19 19 0 0 1 5.3-2.329 20.72 20.72 0 0 1 22.237 8.243 19.17 19.17 0 0 1 3.277 14.503 18 18 0 0 1-.624 2.435l-.49 1.498-1.337-.98a33.6 33.6 0 0 0-10.203-5.1l-.97-.294.09-.968a5.86 5.86 0 0 0-1.052-3.878 6.24 6.24 0 0 0-6.696-2.485 5.8 5.8 0 0 0-1.602.704L37.73 51.72a5.42 5.42 0 0 0-2.449 3.63 5.79 5.79 0 0 0 .986 4.372 6.24 6.24 0 0 0 6.698 2.486 5.8 5.8 0 0 0 1.602-.704l9.952-6.342a19 19 0 0 1 5.295-2.328 20.72 20.72 0 0 1 22.237 8.242 19.17 19.17 0 0 1 3.277 14.503 18 18 0 0 1-8.13 12.053l-26.081 16.622a19 19 0 0 1-5.3 2.328" style="fill:#fff"/></svg>

After

Width:  |  Height:  |  Size: 1.5 KiB

View File

@@ -0,0 +1,12 @@
import adapter from '@sveltejs/adapter-auto';
import { vitePreprocess } from '@sveltejs/vite-plugin-svelte';
/** @type {import('@sveltejs/kit').Config} */
const config = {
// Consult https://svelte.dev/docs/kit/integrations
// for more information about preprocessors
preprocess: vitePreprocess(),
kit: { adapter: adapter() }
};
export default config;

View File

@@ -0,0 +1,19 @@
{
"extends": "./.svelte-kit/tsconfig.json",
"compilerOptions": {
"allowJs": true,
"checkJs": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"skipLibCheck": true,
"sourceMap": true,
"strict": true,
"moduleResolution": "bundler"
}
// Path aliases are handled by https://svelte.dev/docs/kit/configuration#alias
// except $lib which is handled by https://svelte.dev/docs/kit/configuration#files
//
// If you want to overwrite includes/excludes, make sure to copy over the relevant includes/excludes
// from the referenced tsconfig.json - TypeScript does not merge them in
}

View File

@@ -0,0 +1,7 @@
import tailwindcss from '@tailwindcss/vite';
import { sveltekit } from '@sveltejs/kit/vite';
import { defineConfig } from 'vite';
export default defineConfig({
plugins: [tailwindcss(), sveltekit()]
});

View File

@@ -0,0 +1,14 @@
{
"name": "@open-archive/types",
"version": "0.1.0",
"private": true,
"main": "dist/index.js",
"types": "dist/index.d.ts",
"scripts": {
"build": "tsc",
"dev": "tsc --watch"
},
"devDependencies": {
"typescript": "^5.0.0"
}
}

View File

@@ -0,0 +1,7 @@
export type UserRole = 'Admin' | 'Auditor' | 'EndUser';
export interface User {
id: string;
email: string;
role: UserRole;
}

View File

@@ -0,0 +1,11 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"composite": true,
"declaration": true
},
"include": ["src/**/*.ts"],
"exclude": ["node_modules", "dist"]
}

2
pnpm-workspace.yaml Normal file
View File

@@ -0,0 +1,2 @@
packages:
- 'packages/*'

13
tsconfig.base.json Normal file
View File

@@ -0,0 +1,13 @@
{
"compilerOptions": {
"target": "ESNext",
"module": "ESNext",
"moduleResolution": "node",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"isolatedModules": true
}
}