mapleopentech/monorepo
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Initial commit: Open sourcing all of the Maple Open Technologies code.

Browse source
This commit is contained in:
Bartlomiej Mika 2025-12-02 14:33:08 -05:00
commit 755d54a99d
2010 changed files with 448675 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

512
cloud/infrastructure/production/setup/00-multi-app-architecture.md Normal file
View file

@ -0,0 +1,512 @@
# Multi-Application Architecture & Naming Conventions
**Audience**: DevOps Engineers, Infrastructure Team, Developers
**Status**: Architecture Reference Document
**Last Updated**: November 2025
---
## Overview
This document defines the **multi-application architecture** for Maple Open Technologies production infrastructure. The infrastructure is designed to support **multiple independent applications** (MaplePress, MapleFile, mapleopentech) sharing common infrastructure (Cassandra, Redis, Meilisearch) while maintaining clear boundaries and isolation.
---
## Architecture Principles
### 1. Shared Infrastructure, Isolated Applications
```
┌─────────────────────────────────────────────────────────────────┐
│ SHARED INFRASTRUCTURE │
│ (Used by ALL apps: MaplePress, MapleFile, mapleopentech) │
├─────────────────────────────────────────────────────────────────┤
│ Infrastructure Workers (1-5): │
│ - Manager Node (worker-1): Redis │
│ - Cassandra Cluster (workers 2,3,4) │
│ - Meilisearch (worker 5) │
│ │
│ Networks: │
│ - maple-private-prod (databases, cache, search) │
│ - maple-public-prod (reverse proxies + backends) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ APPLICATION: MAPLEPRESS │
├─────────────────────────────────────────────────────────────────┤
│ Worker 6 - MaplePress Backend + Proxy: │
│ Stack: maplepress │
│ │
│ Service: maplepress_backend │
│ Hostname: maplepress-backend │
│ Port: 8000 │
│ Networks: maple-private-prod + maple-public-prod │
│ Connects to: Cassandra, Redis, Meilisearch, Spaces │
│ │
│ Service: maplepress_backend-caddy │
│ Hostname: caddy │
│ Domain: getmaplepress.ca (API) │
│ Proxies to: maplepress-backend:8000 │
│ │
│ Worker 7 - MaplePress Frontend: │
│ Stack: maplepress-frontend │
│ Service: maplepress-frontend_caddy │
│ Hostname: frontend-caddy │
│ Domain: getmaplepress.com (Web UI) │
│ Serves: /var/www/maplepress-frontend/ │
│ Calls: https://getmaplepress.ca (backend API) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ APPLICATION: MAPLEFILE (Future) │
├─────────────────────────────────────────────────────────────────┤
│ Worker 8 - MapleFile Backend + Proxy: │
│ Stack: maplefile │
│ │
│ Service: maplefile_backend │
│ Hostname: maplefile-backend │
│ Port: 8000 │
│ Networks: maple-private-prod + maple-public-prod │
│ Connects to: Cassandra, Redis, Meilisearch, Spaces │
│ │
│ Service: maplefile_backend-caddy │
│ Hostname: maplefile-backend-caddy │
│ Domain: maplefile.ca (API) │
│ Proxies to: maplefile-backend:8000 │
│ │
│ Worker 9 - MapleFile Frontend: │
│ Stack: maplefile-frontend │
│ Service: maplefile-frontend_caddy │
│ Hostname: maplefile-frontend-caddy │
│ Domain: maplefile.com (Web UI) │
│ Serves: /var/www/maplefile-frontend/ │
│ Calls: https://maplefile.ca (backend API) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ APPLICATION: mapleopentech (Future) │
├─────────────────────────────────────────────────────────────────┤
│ Worker 10 - mapleopentech Backend + Proxy: │
│ Stack: mapleopentech │
│ │
│ Service: mapleopentech_backend │
│ Hostname: mapleopentech-backend │
│ Port: 8000 │
│ Networks: maple-private-prod + maple-public-prod │
│ Connects to: Cassandra, Redis, Meilisearch, Spaces │
│ │
│ Service: mapleopentech_backend-caddy │
│ Hostname: mapleopentech-backend-caddy │
│ Domain: api.mapleopentech.io (API) │
│ Proxies to: mapleopentech-backend:8000 │
│ │
│ Worker 11 - mapleopentech Frontend: │
│ Stack: mapleopentech-frontend │
│ Service: mapleopentech-frontend_caddy │
│ Hostname: mapleopentech-frontend-caddy │
│ Domain: mapleopentech.io (Web UI) │
│ Serves: /var/www/mapleopentech-frontend/ │
│ Calls: https://api.mapleopentech.io (backend API) │
└─────────────────────────────────────────────────────────────────┘
```
---
## Naming Conventions
### Pattern: **Option C - Hybrid Stacks**
**Strategy**:
- Backend + Backend Caddy in **one stack** (deployed together)
- Frontend Caddy in **separate stack** (independent deployment)
**Why this pattern?**
- Backend and its reverse proxy are tightly coupled → deploy together
- Frontend is independent → deploy separately
- Avoids redundant naming like `maplepress-backend_backend`
- Clean service names: `maplepress_backend`, `maplepress_backend-caddy`, `maplepress-frontend_caddy`
### Stack Names
| Application | Stack Name | Services in Stack | Purpose |
|-------------|-----------------------|---------------------------------|--------------------------------------|
| MaplePress | `maplepress` | `backend`, `backend-caddy` | Backend API + reverse proxy |
| MaplePress | `maplepress-frontend` | `caddy` | Frontend static files |
| MapleFile | `maplefile` | `backend`, `backend-caddy` | Backend API + reverse proxy |
| MapleFile | `maplefile-frontend` | `caddy` | Frontend static files |
| mapleopentech | `mapleopentech` | `backend`, `backend-caddy` | Backend API + reverse proxy |
| mapleopentech | `mapleopentech-frontend` | `caddy` | Frontend static files |
### Service Names (Docker Auto-Generated)
Docker Swarm automatically creates service names from: `{stack-name}_{service-name}`
| Stack Name | Service in YAML | Full Service Name | Purpose |
|-----------------------|------------------|-----------------------------|----------------------------------|
| `maplepress` | `backend` | `maplepress_backend` | Go backend API |
| `maplepress` | `backend-caddy` | `maplepress_backend-caddy` | Backend reverse proxy |
| `maplepress-frontend` | `caddy` | `maplepress-frontend_caddy` | Frontend static file server |
| `maplefile` | `backend` | `maplefile_backend` | Go backend API |
| `maplefile` | `backend-caddy` | `maplefile_backend-caddy` | Backend reverse proxy |
| `maplefile-frontend` | `caddy` | `maplefile-frontend_caddy` | Frontend static file server |
**View services:**
```bash
docker service ls
# Output:
# maplepress_backend 1/1
# maplepress_backend-caddy 1/1
# maplepress-frontend_caddy 1/1
# maplefile_backend 1/1
# maplefile_backend-caddy 1/1
# maplefile-frontend_caddy 1/1
```
### Hostnames (DNS Resolution Within Networks)
Hostnames are defined in the stack YAML (`hostname: ...`) and used for container-to-container communication.
| Application | Component | Hostname | Used By |
|-------------|-----------------|-------------------------------|----------------------------------|
| MaplePress | Backend | `maplepress-backend` | Caddy proxy, other services |
| MaplePress | Backend Caddy | `caddy` | Internal reference (rarely used) |
| MaplePress | Frontend Caddy | `frontend-caddy` | Internal reference (rarely used) |
| MapleFile | Backend | `maplefile-backend` | Caddy proxy, other services |
| MapleFile | Backend Caddy | `caddy` | Internal reference (rarely used) |
| MapleFile | Frontend Caddy | `frontend-caddy` | Internal reference (rarely used) |
**Example - Caddyfile for MaplePress backend:**
```caddy
getmaplepress.ca www.getmaplepress.ca {
reverse_proxy maplepress-backend:8000 # Uses hostname, not service name
}
```
**Example - Caddyfile for MapleFile backend:**
```caddy
maplefile.ca www.maplefile.ca {
reverse_proxy maplefile-backend:8000
}
```
### Docker Configs (Auto-Generated with Stack Prefix)
| Stack Name | Config in YAML | Full Config Name |
|-------------------------|----------------|-------------------------------------|
| `maplepress` | `caddyfile` | `maplepress_caddyfile` |
| `maplepress-frontend` | `caddyfile` | `maplepress-frontend_caddyfile` |
| `maplefile` | `caddyfile` | `maplefile_caddyfile` |
| `maplefile-frontend` | `caddyfile` | `maplefile-frontend_caddyfile` |
**View configs:**
```bash
docker config ls
# Output:
# maplepress_caddyfile
# maplepress-frontend_caddyfile
# maplefile_caddyfile
# maplefile-frontend_caddyfile
```
### File Paths
| Application | Component | Path |
|-------------|-----------|---------------------------------------|
| MaplePress | Frontend | `/var/www/maplepress-frontend/` |
| MaplePress | Backend | `/var/www/monorepo/cloud/mapleopentech-backend/` |
| MapleFile | Frontend | `/var/www/maplefile-frontend/` |
| MapleFile | Backend | `/var/www/monorepo/cloud/mapleopentech-backend/` |
---
## Resource Allocation
### Workers 1-5: Shared Infrastructure (ALL Apps)
| Worker | Role | Services | Shared By |
|--------|-----------------------|---------------------------------------|----------------|
| 1 | Manager + Redis | Swarm manager, Redis cache | All apps |
| 2 | Cassandra Node 1 | cassandra-1 | All apps |
| 3 | Cassandra Node 2 | cassandra-2 | All apps |
| 4 | Cassandra Node 3 | cassandra-3 | All apps |
| 5 | Meilisearch | meilisearch (full-text search) | All apps |
### Workers 6-7: MaplePress Application
| Worker | Role | Services |
|--------|----------------------------|---------------------------------------------|
| 6 | MaplePress Backend + Proxy | maplepress_backend, maplepress_backend-caddy |
| 7 | MaplePress Frontend | maplepress-frontend_caddy |
### Workers 8-9: MapleFile Application (Future)
| Worker | Role | Services |
|--------|---------------------------|--------------------------------------------|
| 8 | MapleFile Backend + Proxy | maplefile_backend, maplefile_backend-caddy |
| 9 | MapleFile Frontend | maplefile-frontend_caddy |
### Workers 10-11: mapleopentech Application (Future)
| Worker | Role | Services |
|--------|----------------------------|---------------------------------------------|
| 10 | mapleopentech Backend + Proxy | mapleopentech_backend, mapleopentech_backend-caddy |
| 11 | mapleopentech Frontend | mapleopentech-frontend_caddy |
---
## Network Topology
### maple-private-prod (Shared by ALL Apps)
**Purpose**: Private backend services - databases, cache, search
**Services**:
- Cassandra cluster (3 nodes)
- Redis
- Meilisearch
- **All backend services** (maplepress-backend, maplefile-backend, mapleopentech-backend)
**Security**: No ingress ports, no internet access, internal-only
### maple-public-prod (Per-App Reverse Proxies + Backends)
**Purpose**: Internet-facing services - reverse proxies and backends
**Services**:
- **All backend services** (join both networks)
- **All Caddy reverse proxies** (backend + frontend)
**Security**: Ports 80/443 exposed on workers running Caddy
---
## Deployment Commands
### MaplePress
```bash
# Backend + Backend Caddy (deployed together in one stack)
docker stack deploy -c maplepress-stack.yml maplepress
# Frontend (deployed separately)
docker stack deploy -c maplepress-frontend-stack.yml maplepress-frontend
```
### MapleFile (Future)
```bash
# Backend + Backend Caddy (deployed together in one stack)
docker stack deploy -c maplefile-stack.yml maplefile
# Frontend (deployed separately)
docker stack deploy -c maplefile-frontend-stack.yml maplefile-frontend
```
### mapleopentech (Future)
```bash
# Backend + Backend Caddy (deployed together in one stack)
docker stack deploy -c mapleopentech-stack.yml mapleopentech
# Frontend (deployed separately)
docker stack deploy -c mapleopentech-frontend-stack.yml mapleopentech-frontend
```
---
## Verification Commands
### List All Stacks
```bash
docker stack ls
# Expected output:
# NAME SERVICES
# cassandra 3
# maplepress 2 (backend + backend-caddy)
# maplepress-frontend 1 (frontend caddy)
# maplefile 2 (future)
# maplefile-frontend 1 (future)
# meilisearch 1
# redis 1
```
### List All Services
```bash
docker service ls | sort
# Expected output (partial):
# cassandra_cassandra-1 1/1
# cassandra_cassandra-2 1/1
# cassandra_cassandra-3 1/1
# maplepress_backend 1/1
# maplepress_backend-caddy 1/1
# maplepress-frontend_caddy 1/1
# meilisearch_meilisearch 1/1
# redis_redis 1/1
```
### List All Configs
```bash
docker config ls
# Expected output:
# maplepress_caddyfile
# maplepress-frontend_caddyfile
```
---
## Adding a New Application
To add a new application (e.g., "MaplePortal"):
### 1. Update .env.template
```bash
# Add new section
# ==============================================================================
# MAPLEPORTAL APPLICATION
# ==============================================================================
# Backend Configuration
MAPLEPORTAL_BACKEND_DOMAIN=api.mapleportal.io
MAPLEPORTAL_SPACES_BUCKET=mapleportal-prod
MAPLEPORTAL_JWT_SECRET=CHANGEME
MAPLEPORTAL_IP_ENCRYPTION_KEY=CHANGEME
# Frontend Configuration
MAPLEPORTAL_FRONTEND_DOMAIN=mapleportal.io
MAPLEPORTAL_FRONTEND_API_URL=https://api.mapleportal.io
```
### 2. Create New Workers
```bash
# Worker 12 - Backend + Backend Caddy
# Worker 13 - Frontend Caddy
```
### 3. Follow Naming Convention
- Stack names: `mapleportal` (backend + backend-caddy), `mapleportal-frontend`
- Service names: `mapleportal_backend`, `mapleportal_backend-caddy`, `mapleportal-frontend_caddy`
- Hostnames: `mapleportal-backend`, `mapleportal-backend-caddy`, `mapleportal-frontend-caddy`
- Domains: `api.mapleportal.io` (backend), `mapleportal.io` (frontend)
- Paths: `/var/www/mapleportal-frontend/`
### 4. Deploy Services
```bash
# Backend + backend-caddy in one stack
docker stack deploy -c mapleportal-stack.yml mapleportal
# Frontend in separate stack
docker stack deploy -c mapleportal-frontend-stack.yml mapleportal-frontend
```
---
## Benefits of This Architecture
### 1. Clear Separation
- Each app has dedicated workers and services
- No naming conflicts between apps
- Easy to identify which services belong to which app
### 2. Shared Infrastructure Efficiency
- Single Cassandra cluster serves all apps
- Single Redis instance (or sharded by app)
- Single Meilisearch instance with app-prefixed indexes
- Cost savings: 5 workers for infrastructure vs 15+ if each app had its own
### 3. Independent Scaling
- Scale MaplePress without affecting MapleFile
- Deploy new apps without touching existing ones
- Remove apps without impacting infrastructure
### 4. Operational Clarity
```bash
# View only MaplePress services
docker service ls | grep maplepress
# View only MapleFile services
docker service ls | grep maplefile
# Restart MaplePress backend
docker service update --force maplepress_backend
# Remove MapleFile entirely (if needed)
docker stack rm maplefile
docker stack rm maplefile-frontend
```
### 5. Developer Friendly
- Developers instantly know which app they're working with
- No ambiguous "backend" or "frontend" names
- Service discovery is intuitive: `maplepress-backend:8000`
---
## Migration Checklist (For Existing Deployments)
If you deployed with old naming (`caddy`, `maplepress`, `frontend-caddy`), migrate like this:
### Step 1: Update Configuration Files Locally
```bash
cd ~/monorepo/cloud/infrastructure/production
# Update all YAML files to use new naming
# - maplepress → maplepress-backend
# - caddy → maplepress-backend-caddy
# - frontend-caddy → maplepress-frontend-caddy
# Update Caddyfiles to use new hostnames
# - backend:8000 → maplepress-backend:8000
```
### Step 2: Remove Old Stacks
```bash
# On manager node
docker stack rm maplepress
docker stack rm caddy
docker stack rm frontend-caddy
# Wait for cleanup
sleep 10
# Remove old configs
docker config rm caddy_caddyfile
docker config rm frontend-caddy_caddyfile
```
### Step 3: Deploy New Stacks
```bash
# Deploy with new names (Option C naming)
docker stack deploy -c maplepress-stack.yml maplepress
docker stack deploy -c maplepress-frontend-stack.yml maplepress-frontend
```
### Step 4: Verify
```bash
docker service ls
# Should show:
# maplepress_backend
# maplepress_backend-caddy
# maplepress-frontend_caddy
docker config ls
# Should show:
# maplepress_caddyfile
# maplepress-frontend_caddyfile
```
---
**Last Updated**: November 2025
**Maintained By**: Infrastructure Team
**Status**: Production Standard - Follow for All New Applications