monorepo/cloud/maplefile-backend/README.md

# 🚀 MapleFile Backend

> Secure, end-to-end encrypted file storage backend - Zero-knowledge architecture built with Go.

MapleFile provides military-grade file encryption with client-side E2EE (End-to-End Encryption). Features include collection-based organization, granular sharing permissions, JWT authentication, and S3-compatible object storage. Your files are encrypted on your device before reaching our servers - we never see your data.

## 📋 Prerequisites

**⚠️ Required:** You must have the infrastructure running first.

If you haven't set up the infrastructure yet:
1. Go to [`../infrastructure/README.md`](../infrastructure/README.md)
2. Follow the setup instructions
3. Come back here once infrastructure is running

**Verify infrastructure is healthy:**
```bash
cd cloud/infrastructure/development
task dev:status
# All services should show (healthy)
```

## 🏁 Getting Started

### Installation

```bash
# From the monorepo root:
cd cloud/maplefile-backend

# Create environment file:
cp .env.sample .env

# Start the backend:
task dev
```

The backend runs at **http://localhost:8000**

### Verify Installation

Open a **new terminal** (leave `task dev` running):

```bash
curl http://localhost:8000/health
# Should return: {"status":"healthy","service":"maplefile-backend","di":"Wire"}
```

> **Note:** Your first terminal shows backend logs. Keep it running and use a second terminal for testing.

## 💻 Developing

### Initial Configuration

**Environment Files:**
- **`.env.sample`** - Template with defaults (committed to git)
- **`.env`** - Your local configuration (git-ignored, created from `.env.sample`)
- Use **only `.env`** for configuration (docker-compose loads this file)

The `.env` file defaults work for Docker development. **Optional:** Change `BACKEND_APP_JWT_SECRET` to a random string (use a password generator).

### Running in Development Mode

```bash
# Start backend with hot-reload
task dev

# View logs (in another terminal)
docker logs -f maplefile-backend-dev

# Stop backend
task dev:down
# Or press Ctrl+C in the task dev terminal
```

**What happens when you run `task dev`:**
- Docker starts the backend container
- Auto-migrates database tables
- Starts HTTP server on port 8000
- Enables hot-reload (auto-restarts on code changes)

Wait for: `✅ Database migrations completed successfully` in the logs

### Daily Workflow

```bash
# Morning - check infrastructure (from monorepo root)
cd cloud/infrastructure/development && task dev:status

# Start backend (from monorepo root)
cd cloud/maplefile-backend && task dev

# Make code changes - backend auto-restarts

# Stop backend when done
# Press Ctrl+C
```

### Testing

```bash
# Run all tests
task test

# Code quality checks
task format  # Format code
task lint    # Run linters
```

### Database Operations

**View database:**
```bash
# From monorepo root
cd cloud/infrastructure/development
task cql

# Inside cqlsh:
USE maplefile;
DESCRIBE TABLES;
SELECT * FROM users_by_id;
```

**Reset database (⚠️ deletes all data):**
```bash
task db:clear
```

## 🔧 Usage

### Testing the API

Create a test user to verify the backend works:

**1. Register a user:**
```bash
curl -X POST http://localhost:8000/api/v1/auth/register \
  -H "Content-Type: application/json" \
  -d '{
    "email": "test@example.com",
    "first_name": "Test",
    "last_name": "User",
    "phone": "+1234567890",
    "country": "Canada",
    "timezone": "America/Toronto",
    "salt": "base64-encoded-salt",
    "kdf_algorithm": "argon2id",
    "kdf_iterations": 3,
    "kdf_memory": 65536,
    "kdf_parallelism": 4,
    "kdf_salt_length": 16,
    "kdf_key_length": 32,
    "encryptedMasterKey": "base64-encoded-encrypted-master-key",
    "publicKey": "base64-encoded-public-key",
    "encryptedPrivateKey": "base64-encoded-encrypted-private-key",
    "encryptedRecoveryKey": "base64-encoded-encrypted-recovery-key",
    "masterKeyEncryptedWithRecoveryKey": "base64-encoded-master-key-encrypted-with-recovery",
    "agree_terms_of_service": true,
    "agree_promotions": false,
    "agree_to_tracking_across_third_party_apps_and_services": false
  }'
```

> **Note:** MapleFile uses end-to-end encryption. The frontend (maplefile-frontend) handles all cryptographic operations. For manual API testing, you'll need to generate valid encryption keys using libsodium. See the frontend registration implementation for reference.

**Response:**
```json
{
  "message": "Registration successful. Please check your email to verify your account.",
  "user_id": "uuid-here"
}
```

**2. Verify email:**
Check your email for the verification code, then:
```bash
curl -X POST http://localhost:8000/api/v1/auth/verify-email \
  -H "Content-Type: application/json" \
  -d '{
    "email": "test@example.com",
    "verification_code": "123456"
  }'
```

**3. Login:**
```bash
curl -X POST http://localhost:8000/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{
    "email": "test@example.com"
  }'
```

Check your email for the OTP (One-Time Password), then complete login:
```bash
curl -X POST http://localhost:8000/api/v1/auth/login/verify-otp \
  -H "Content-Type: application/json" \
  -d '{
    "email": "test@example.com",
    "otp": "your-otp-code",
    "encrypted_challenge": "base64-encoded-challenge-response"
  }'
```

**Response:**
```json
{
  "access_token": "eyJhbGci...",
  "refresh_token": "eyJhbGci...",
  "access_expiry": "2025-11-12T13:00:00Z",
  "refresh_expiry": "2025-11-19T12:00:00Z"
}
```

Save the `access_token` from the response:
```bash
export TOKEN="eyJhbGci...your-access-token-here"
```

**4. Get your profile:**
```bash
curl http://localhost:8000/api/v1/me \
  -H "Authorization: JWT $TOKEN"
```

**5. Get dashboard:**
```bash
curl http://localhost:8000/api/v1/dashboard \
  -H "Authorization: JWT $TOKEN"
```

**6. Create a collection (folder):**
```bash
curl -X POST http://localhost:8000/api/v1/collections \
  -H "Content-Type: application/json" \
  -H "Authorization: JWT $TOKEN" \
  -d '{
    "name": "My Documents",
    "description": "Personal documents",
    "collection_type": "folder",
    "encrypted_collection_key": "base64-encoded-encrypted-key"
  }'
```

**7. Upload a file:**
```bash
# First, get a presigned URL
curl -X POST http://localhost:8000/api/v1/files/presigned-url \
  -H "Content-Type: application/json" \
  -H "Authorization: JWT $TOKEN" \
  -d '{
    "file_name": "document.pdf",
    "file_size": 1024000,
    "mime_type": "application/pdf",
    "collection_id": "your-collection-id"
  }'

# Upload the encrypted file to the presigned URL (using the URL from response)
curl -X PUT "presigned-url-here" \
  --upload-file your-encrypted-file.enc

# Report upload completion
curl -X POST http://localhost:8000/api/v1/files/upload-complete \
  -H "Content-Type: application/json" \
  -H "Authorization: JWT $TOKEN" \
  -d '{
    "file_id": "file-id-from-presigned-response",
    "status": "completed"
  }'
```

### Frontend Integration

**Access the frontend:**
- URL: http://localhost:5173
- The frontend handles all encryption/decryption automatically
- See [`../../web/maplefile-frontend/README.md`](../../web/maplefile-frontend/README.md)

**Key Features:**
- 🔐 **Client-side encryption** - Files encrypted before upload
- 🔑 **E2EE Key Chain** - Password → KEK → Master Key → Collection Keys → File Keys
- 📁 **Collections** - Organize files in encrypted folders
- 🤝 **Sharing** - Share collections with read-only, read-write, or admin permissions
- 🔄 **Sync modes** - Cloud-only, local-only, or hybrid storage

**Next steps:**
- Frontend setup: [`../../web/maplefile-frontend/README.md`](../../web/maplefile-frontend/README.md)
- Complete API documentation: See API endpoints in code

## ⚙️ Configuration

### Environment Variables

Key variables in `.env`:

| Variable | Default | Description |
|----------|---------|-------------|
| `BACKEND_APP_JWT_SECRET` | `change-me-in-production` | Secret for JWT token signing |
| `BACKEND_APP_SERVER_PORT` | `8000` | HTTP server port |
| `BACKEND_DB_HOSTS` | `cassandra-1,cassandra-2,cassandra-3` | Cassandra cluster nodes |
| `BACKEND_CACHE_HOST` | `redis` | Redis cache host |
| `BACKEND_MAPLEFILE_S3_ENDPOINT` | `http://seaweedfs:8333` | S3 storage URL |
| `BACKEND_MAPLEFILE_S3_BUCKET` | `maplefile` | S3 bucket name |

**Docker vs Local:**
- Docker: Uses container names (`cassandra-1`, `redis`, `seaweedfs`)
- Local: Change to `localhost`

See `.env.sample` for complete documentation.

### Task Commands

| Command | Description |
|---------|-------------|
| `task dev` | Start backend (auto-migrate + hot-reload) |
| `task dev:down` | Stop backend |
| `task test` | Run tests |
| `task format` | Format code |
| `task lint` | Run linters |
| `task db:clear` | Reset database (⚠️ deletes data) |
| `task migrate:up` | Manual migration |
| `task build` | Build binary |

## 🔍 Troubleshooting

### Backend won't start - "connection refused"

**Error:** `dial tcp 127.0.0.1:9042: connect: connection refused`

**Cause:** `.env` file has `localhost` instead of container names.

**Fix:**
```bash
cd cloud/maplefile-backend
rm .env
cp .env.sample .env
task dev
```

### Infrastructure not running

**Error:** Cassandra or Redis not available

**Fix:**
```bash
cd cloud/infrastructure/development
task dev:start
task dev:status  # Wait until all show (healthy)
```

### Port 8000 already in use

**Fix:**
```bash
lsof -i :8000  # Find what's using the port
# Stop the other service, or change BACKEND_APP_SERVER_PORT in .env
```

### Token expired (401 errors)

JWT tokens expire after 60 minutes. Re-run the [login steps](#testing-the-api) to get a new token.

### Database keyspace not found

**Error:** `Keyspace 'maplefile' does not exist` or `failed to create user`

**Cause:** The Cassandra keyspace hasn't been created yet. This is a one-time infrastructure setup.

**Fix:**
```bash
# Initialize the keyspace (one-time setup)
cd cloud/infrastructure/development

# Find Cassandra container
export CASSANDRA_CONTAINER=$(docker ps --filter "name=cassandra" -q | head -1)

# Create keyspace
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "
CREATE KEYSPACE IF NOT EXISTS maplefile
WITH replication = {
    'class': 'SimpleStrategy',
    'replication_factor': 3
};"

# Verify keyspace exists
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "DESCRIBE KEYSPACE maplefile;"

# Restart backend to retry migrations
cd ../../maplefile-backend
task dev:restart
```

**Note:** The backend auto-migrates tables on startup, but expects the keyspace to already exist. This is standard practice - keyspaces are infrastructure setup, not application migrations.

## 🛠️ Technology Stack

- **Go 1.23+** - Programming language
- **Clean Architecture** - Code organization
- **Wire** - Dependency injection (Google's code generation)
- **Cassandra 5.0.4** - Distributed database (3-node cluster)
- **Redis 7** - Caching layer
- **SeaweedFS** - S3-compatible object storage
- **JWT** - User authentication
- **ChaCha20-Poly1305** - Authenticated encryption (client-side)
- **Argon2id** - Password hashing / KDF

## 🌐 Services

When you run MapleFile, these services are available:

| Service | Port | Purpose | Access |
|---------|------|---------|--------|
| MapleFile Backend | 8000 | HTTP API | http://localhost:8000 |
| MapleFile Frontend | 5173 | Web UI | http://localhost:5173 |
| Cassandra | 9042 | Database | `task cql` (from infrastructure dir) |
| Redis | 6379 | Cache | `task redis` (from infrastructure dir) |
| SeaweedFS S3 | 8333 | Object storage | http://localhost:8333 |
| SeaweedFS UI | 9333 | Storage admin | http://localhost:9333 |

## 🏗️ Architecture

### Project Structure

```
maplefile-backend/
├── cmd/              # CLI commands (daemon, migrate, version)
├── config/           # Configuration loading
├── internal/         # Application code
│   ├── app/          # Wire application wiring
│   ├── domain/       # Domain entities
│   │   ├── collection/  # Collections (folders)
│   │   ├── crypto/      # Encryption types
│   │   ├── file/        # File metadata
│   │   ├── user/        # User accounts
│   │   └── ...
│   ├── repo/         # Repository implementations (Cassandra)
│   ├── usecase/      # Use cases / business logic
│   ├── service/      # Service layer
│   └── interface/    # HTTP handlers
│       └── http/     # REST API endpoints
├── pkg/              # Shared infrastructure
│   ├── storage/      # Database, cache, S3, memory
│   ├── security/     # JWT, encryption, password hashing
│   └── emailer/      # Email sending
├── migrations/       # Cassandra schema migrations
└── docs/             # Documentation
```

### Key Features

- **🔐 Zero-Knowledge Architecture**: Files encrypted on client, server never sees plaintext
- **🔑 E2EE Key Chain**: User Password → KEK → Master Key → Collection Keys → File Keys
- **📦 Storage Modes**: `encrypted_only`, `hybrid`, `decrypted_only`
- **🤝 Collection Sharing**: `read_only`, `read_write`, `admin` permissions
- **💾 Two-Tier Caching**: Redis + Cassandra-based cache
- **📊 Storage Quotas**: 10GB default per user
- **🔄 File Versioning**: Soft delete with tombstone tracking

### End-to-End Encryption Flow

```
1. User enters password → Frontend derives KEK (Key Encryption Key)
2. KEK → Encrypts/decrypts Master Key (stored encrypted on server)
3. Master Key → Encrypts/decrypts Collection Keys
4. Collection Key → Encrypts/decrypts File Keys
5. File Key → Encrypts/decrypts actual file content

Server only stores:
- Encrypted Master Key (encrypted with KEK from password)
- Encrypted Collection Keys (encrypted with Master Key)
- Encrypted File Keys (encrypted with Collection Key)
- Encrypted file content (encrypted with File Key)

Server NEVER has access to:
- User's password
- KEK (derived from password on client)
- Decrypted Master Key
- Decrypted Collection Keys
- Decrypted File Keys
- Plaintext file content
```

## 🔗 Links

- **Frontend Application:** [`../../web/maplefile-frontend/README.md`](../../web/maplefile-frontend/README.md)
- **CLI Tool:** [`../../native/desktop/maplefile/README.md`](../../native/desktop/maplefile/README.md)
- **Architecture Details:** [`../../CLAUDE.md`](../../CLAUDE.md)
- **Repository:** [Codeberg - mapleopentech/monorepo](https://codeberg.org/mapleopentech/monorepo)

## 🤝 Contributing

Found a bug? Want a feature to improve MapleFile? Please create an [issue](https://codeberg.org/mapleopentech/monorepo/issues/new).

## 📝 License

This application is licensed under the [**GNU Affero General Public License v3.0**](https://opensource.org/license/agpl-v3). See [LICENSE](../../LICENSE) for more information.