mapleopentech/monorepo
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Refactored.

Browse source
This commit is contained in:
Bartlomiej Mika 2025-12-02 22:48:40 -05:00
parent f4a49ad4b9
commit 9dad75464b
37 changed files with 667 additions and 247 deletions
Download patch file Download diff file Expand all files Collapse all files

14
cloud/infrastructure/production/setup/00-multi-app-architecture.md
View file

@ -27,8 +27,8 @@ This document defines the **multi-application architecture** for Maple Open Tech
│ - Meilisearch (worker 5) │
│ │
│ Networks: │
│ - maple-private-prod (databases, cache, search) │
│ - maple-public-prod (reverse proxies + backends) │
│ - mapleopentech-private-prod (databases, cache, search) │
│ - mapleopentech-public-prod (reverse proxies + backends) │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
@ -40,7 +40,7 @@ This document defines the **multi-application architecture** for Maple Open Tech
│ Service: maplepress_backend │
│ Hostname: maplepress-backend │
│ Port: 8000 │
│ Networks: maple-private-prod + maple-public-prod │
│ Networks: mapleopentech-private-prod + mapleopentech-public-prod │
│ Connects to: Cassandra, Redis, Meilisearch, Spaces │
│ │
│ Service: maplepress_backend-caddy │
@ -66,7 +66,7 @@ This document defines the **multi-application architecture** for Maple Open Tech
│ Service: maplefile_backend │
│ Hostname: maplefile-backend │
│ Port: 8000 │
│ Networks: maple-private-prod + maple-public-prod │
│ Networks: mapleopentech-private-prod + mapleopentech-public-prod │
│ Connects to: Cassandra, Redis, Meilisearch, Spaces │
│ │
│ Service: maplefile_backend-caddy │
@ -92,7 +92,7 @@ This document defines the **multi-application architecture** for Maple Open Tech
│ Service: mapleopentech_backend │
│ Hostname: mapleopentech-backend │
│ Port: 8000 │
│ Networks: maple-private-prod + maple-public-prod │
│ Networks: mapleopentech-private-prod + mapleopentech-public-prod │
│ Connects to: Cassandra, Redis, Meilisearch, Spaces │
│ │
│ Service: mapleopentech_backend-caddy │
@ -256,7 +256,7 @@ docker config ls
## Network Topology
### maple-private-prod (Shared by ALL Apps)
### mapleopentech-private-prod (Shared by ALL Apps)
**Purpose**: Private backend services - databases, cache, search
@ -268,7 +268,7 @@ docker config ls
**Security**: No ingress ports, no internet access, internal-only
### maple-public-prod (Per-App Reverse Proxies + Backends)
### mapleopentech-public-prod (Per-App Reverse Proxies + Backends)
**Purpose**: Internet-facing services - reverse proxies and backends

48
cloud/infrastructure/production/setup/00-network-architecture.md
View file

@ -15,7 +15,7 @@ We use a **multi-network architecture** following industry best practices for se
│ Docker Swarm Cluster │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ maple-private-prod (Overlay Network) │ │
│ │ mapleopentech-private-prod (Overlay Network) │ │
│ │ No Internet Access | Internal Services Only │ │
│ │ SHARED by ALL applications │ │
│ ├────────────────────────────────────────────────────────────┤ │
@ -31,7 +31,7 @@ We use a **multi-network architecture** following industry best practices for se
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ maple-public-prod (Overlay Network) │ │
│ │ mapleopentech-public-prod (Overlay Network) │ │
│ │ Internet-Facing | Public Services │ │
│ ├────────────────────────────────────────────────────────────┤ │
│ │ Reverse Proxies (Caddy - ports 80/443): │ │
@ -48,14 +48,14 @@ We use a **multi-network architecture** following industry best practices for se
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ Note: Application backends join BOTH networks: │
│ - Receive requests from Caddy on maple-public-prod │
│ - Access databases/cache on maple-private-prod │
│ - Receive requests from Caddy on mapleopentech-public-prod │
│ - Access databases/cache on mapleopentech-private-prod │
└─────────────────────────────────────────────────────────────────┘
```
## Networks Explained
### 1. `maple-private-prod` (Current)
### 1. `mapleopentech-private-prod` (Current)
**Purpose**: Backend services that should NEVER be exposed to the internet.
@ -86,7 +86,7 @@ cassandra-2:9042
cassandra-3:9042
```
### 2. `maple-public-prod` (Current - In Use)
### 2. `mapleopentech-public-prod` (Current - In Use)
**Purpose**: Internet-facing services that handle external traffic.
@ -110,9 +110,9 @@ cassandra-3:9042
**Routing Flow:**
```
Internet → Caddy Reverse Proxy (maple-public-prod)
→ Application Backend (maple-public-prod + maple-private-prod)
→ Databases/Cache (maple-private-prod only)
Internet → Caddy Reverse Proxy (mapleopentech-public-prod)
→ Application Backend (mapleopentech-public-prod + mapleopentech-private-prod)
→ Databases/Cache (mapleopentech-private-prod only)
Example (MaplePress):
https://getmaplepress.ca → maplepress-backend-caddy
@ -157,19 +157,19 @@ Both networks are created and in use:
docker network create \
--driver overlay \
--attachable \
maple-private-prod
mapleopentech-private-prod
# Create public network (done in 06_caddy.md - used by reverse proxies)
docker network create \
--driver overlay \
--attachable \
maple-public-prod
mapleopentech-public-prod
# Verify both exist
docker network ls | grep maple
# Should show:
# maple-private-prod
# maple-public-prod
# mapleopentech-private-prod
# mapleopentech-public-prod
```
### Multi-App Pattern
@ -183,7 +183,7 @@ docker network ls | grep maple
### Go Backend Connecting to Services
**On `maple-private-prod` network:**
**On `mapleopentech-private-prod` network:**
```go
// Redis connection
@ -206,16 +206,16 @@ services:
backend:
image: your-backend:latest
networks:
- maple-private-prod # Access to databases
- maple-public-prod # Receive HTTP requests (when deployed)
- mapleopentech-private-prod # Access to databases
- mapleopentech-public-prod # Receive HTTP requests (when deployed)
environment:
- REDIS_HOST=redis
- CASSANDRA_HOSTS=cassandra-1,cassandra-2,cassandra-3
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
maple-public-prod:
mapleopentech-public-prod:
external: true
```
@ -256,7 +256,7 @@ docker service inspect your_service --format '{{.Spec.TaskTemplate.Networks}}'
### Test Connectivity Between Networks
```bash
# From a container on maple-private-prod
# From a container on mapleopentech-private-prod
docker exec -it <container> ping redis
docker exec -it <container> nc -zv cassandra-1 9042
@ -266,19 +266,19 @@ docker exec -it <container> nc -zv cassandra-1 9042
### View All Services on a Network
```bash
docker network inspect maple-private-prod --format '{{range .Containers}}{{.Name}} {{end}}'
docker network inspect mapleopentech-private-prod --format '{{range .Containers}}{{.Name}} {{end}}'
```
## Migration Path
### Current Status
- ✅ `maple-private-prod` created
- ✅ Cassandra on `maple-private-prod`
- ✅ Redis on `maple-private-prod`
- ✅ `mapleopentech-private-prod` created
- ✅ Cassandra on `mapleopentech-private-prod`
- ✅ Redis on `mapleopentech-private-prod`
- ⏳ Backend deployment (next)
- ⏳ Public network + NGINX (future)
### When to Create `maple-public-prod`
### When to Create `mapleopentech-public-prod`
Create the public network when you're ready to:
1. Deploy NGINX reverse proxy

30
cloud/infrastructure/production/setup/02_cassandra.md
View file

@ -50,7 +50,7 @@ Cassandra Cluster (NEW):
### Cassandra Configuration
- **Version**: Cassandra 5.0.4
- **Cluster Name**: maple-private-prod-cluster
- **Cluster Name**: mapleopentech-private-prod-cluster
- **Replication Factor**: 3 (each data stored on all 3 nodes)
- **Data Center**: datacenter1
- **Heap Size**: 512MB (reduced for 2GB RAM constraint)
@ -358,7 +358,7 @@ Copy and paste the following:
version: '3.8'
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
volumes:
@ -371,9 +371,9 @@ services:
image: cassandra:5.0.4
hostname: cassandra-1
networks:
- maple-private-prod
- mapleopentech-private-prod
environment:
- CASSANDRA_CLUSTER_NAME=maple-private-prod-cluster
- CASSANDRA_CLUSTER_NAME=mapleopentech-private-prod-cluster
- CASSANDRA_DC=datacenter1
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
- CASSANDRA_SEEDS=cassandra-1,cassandra-2,cassandra-3
@ -401,9 +401,9 @@ services:
image: cassandra:5.0.4
hostname: cassandra-2
networks:
- maple-private-prod
- mapleopentech-private-prod
environment:
- CASSANDRA_CLUSTER_NAME=maple-private-prod-cluster
- CASSANDRA_CLUSTER_NAME=mapleopentech-private-prod-cluster
- CASSANDRA_DC=datacenter1
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
- CASSANDRA_SEEDS=cassandra-1,cassandra-2,cassandra-3
@ -431,9 +431,9 @@ services:
image: cassandra:5.0.4
hostname: cassandra-3
networks:
- maple-private-prod
- mapleopentech-private-prod
environment:
- CASSANDRA_CLUSTER_NAME=maple-private-prod-cluster
- CASSANDRA_CLUSTER_NAME=mapleopentech-private-prod-cluster
- CASSANDRA_DC=datacenter1
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
- CASSANDRA_SEEDS=cassandra-1,cassandra-2,cassandra-3
@ -460,19 +460,19 @@ services:
### Step 4: Create Shared Overlay Network
Before deploying any services, create the shared `maple-private-prod` network that all services will use:
Before deploying any services, create the shared `mapleopentech-private-prod` network that all services will use:
```bash
# Create the maple-private-prod overlay network
# Create the mapleopentech-private-prod overlay network
docker network create \
--driver overlay \
--attachable \
maple-private-prod
mapleopentech-private-prod
# Verify it was created
docker network ls | grep maple-private-prod
docker network ls | grep mapleopentech-private-prod
# Should show:
# abc123... maple-private-prod overlay swarm
# abc123... mapleopentech-private-prod overlay swarm
```
**What is this network for?**
@ -847,7 +847,7 @@ docker exec -it $CONTAINER_ID cqlsh -e "SELECT * FROM test.users;"
```bash
# On your local machine, add:
CASSANDRA_CLUSTER_NAME=maple-private-prod-cluster
CASSANDRA_CLUSTER_NAME=mapleopentech-private-prod-cluster
CASSANDRA_DC=datacenter1
CASSANDRA_REPLICATION_FACTOR=3
@ -1132,7 +1132,7 @@ docker exec -it $(docker ps -q --filter "name=cassandra") nodetool status
```yaml
# In your application stack file:
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
```

46
cloud/infrastructure/production/setup/03_redis.md
View file

@ -7,7 +7,7 @@
**What You'll Build**:
- Single Redis instance on existing worker-1
- Password-protected with Docker secrets
- Private network communication only (maple-private-prod overlay)
- Private network communication only (mapleopentech-private-prod overlay)
- Persistent data with AOF + RDB
- Ready for Go application connections
@ -37,23 +37,23 @@ Docker Swarm Cluster:
│
├── mapleopentech-swarm-worker-1-prod (10.116.0.3)
│ └── Redis (single instance)
│ ├── Network: maple-private-prod (overlay, shared)
│ ├── Network: mapleopentech-private-prod (overlay, shared)
│ ├── Port: 6379 (private only)
│ ├── Auth: Password (Docker secret)
│ └── Data: Persistent volume
│
└── mapleopentech-swarm-worker-2,3,4-prod
└── Cassandra Cluster (3 nodes)
└── Same network: maple-private-prod
└── Same network: mapleopentech-private-prod
Shared Network (maple-private-prod):
Shared Network (mapleopentech-private-prod):
├── All services can communicate
├── Service discovery by name (redis, cassandra-1, etc.)
└── No public internet access
Future Application:
└── mapleopentech-swarm-worker-X-prod
└── Go Backend → Connects to redis:6379 and cassandra:9042 on maple-private-prod
└── Go Backend → Connects to redis:6379 and cassandra:9042 on mapleopentech-private-prod
```
### Redis Configuration
@ -164,7 +164,7 @@ Copy and paste the following:
version: '3.8'
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
volumes:
@ -179,7 +179,7 @@ services:
image: redis:7-alpine
hostname: redis
networks:
- maple-private-prod
- mapleopentech-private-prod
volumes:
- redis-data:/data
secrets:
@ -240,16 +240,16 @@ Save and exit (`:wq` in vi).
### Step 2: Verify Shared Overlay Network
**Check if the maple-private-prod network exists:**
**Check if the mapleopentech-private-prod network exists:**
```bash
docker network ls | grep maple-private-prod
docker network ls | grep mapleopentech-private-prod
```
**You should see:**
```
abc123... maple-private-prod overlay swarm
abc123... mapleopentech-private-prod overlay swarm
```
**If you completed 02_cassandra.md** (Step 4), the network already exists and you're good to go!
@ -257,14 +257,14 @@ abc123... maple-private-prod overlay swarm
**If the network doesn't exist**, create it now:
```bash
# Create the shared maple-private-prod network
# Create the shared mapleopentech-private-prod network
docker network create \
--driver overlay \
--attachable \
maple-private-prod
mapleopentech-private-prod
# Verify it was created
docker network ls | grep maple-private-prod
docker network ls | grep mapleopentech-private-prod
```
**What is this network?**
@ -436,22 +436,22 @@ docker stack deploy -c redis-stack.yml redis
### Problem: Network Not Found During Deployment
**Symptom**: `network "maple-private-prod" is declared as external, but could not be found`
**Symptom**: `network "mapleopentech-private-prod" is declared as external, but could not be found`
**Solution:**
Create the shared `maple-private-prod` network first:
Create the shared `mapleopentech-private-prod` network first:
```bash
# Create the network
docker network create \
--driver overlay \
--attachable \
maple-private-prod
mapleopentech-private-prod
# Verify it exists
docker network ls | grep maple-private-prod
# Should show: maple-private-prod overlay swarm
docker network ls | grep mapleopentech-private-prod
# Should show: mapleopentech-private-prod overlay swarm
# Then deploy Redis
docker stack deploy -c redis-stack.yml redis
@ -487,10 +487,10 @@ docker stack deploy -c redis-stack.yml redis
# Must show: map[redis:true]
```
4. **Verify maple-private-prod network exists:**
4. **Verify mapleopentech-private-prod network exists:**
```bash
docker network ls | grep maple-private-prod
# Should show: maple-private-prod overlay swarm
docker network ls | grep mapleopentech-private-prod
# Should show: mapleopentech-private-prod overlay swarm
```
### Problem: Can't Connect (Authentication Failed)
@ -548,9 +548,9 @@ docker stack deploy -c redis-stack.yml redis
1. **Verify both services on same network:**
```bash
# Check your app is on maple-private-prod network
# Check your app is on mapleopentech-private-prod network
docker service inspect your_app --format '{{.Spec.TaskTemplate.Networks}}'
# Should show maple-private-prod
# Should show mapleopentech-private-prod
```
2. **Test DNS resolution:**

BIN
cloud/infrastructure/production/setup/04_meilisearch.md
View file

Binary file not shown.

32
cloud/infrastructure/production/setup/05_maplepress_backend.md
View file

@ -41,7 +41,7 @@ Internet (HTTPS) → Caddy (worker-6) → Backend (worker-6) → Cassandra/Redis
- **Image**: Ubuntu 24.04 LTS x64
- **Size**: Basic shared CPU, 2 GB / 2 vCPU ($18/mo)
- **Hostname**: `mapleopentech-swarm-worker-6-prod`
- **VPC Network**: Select same VPC as your swarm (maple-vpc-prod)
- **VPC Network**: Select same VPC as your swarm (mapleopentech-vpc-prod)
- **SSH Keys**: Add your SSH key
4. Click **Create Droplet**
5. Wait 1-2 minutes for droplet to provision
@ -349,8 +349,8 @@ dig www.getmaplepress.ca +short
We need two overlay networks for our services:
1. **maple-private-prod** - Backend connects to databases (already exists from guides 02-04)
2. **maple-public-prod** - NGINX and Backend communicate (new)
1. **mapleopentech-private-prod** - Backend connects to databases (already exists from guides 02-04)
2. **mapleopentech-public-prod** - NGINX and Backend communicate (new)
**On manager:**
@ -358,11 +358,11 @@ We need two overlay networks for our services:
ssh dockeradmin@<manager-public-ip>
# Check private network exists
docker network ls | grep maple-private-prod
# Should show: maple-private-prod (created in previous guides)
docker network ls | grep mapleopentech-private-prod
# Should show: mapleopentech-private-prod (created in previous guides)
# Create public network
docker network create --driver overlay --attachable maple-public-prod
docker network create --driver overlay --attachable mapleopentech-public-prod
# Verify both exist
docker network ls | grep maple
@ -371,8 +371,8 @@ docker network ls | grep maple
**Expected output:**
```
abc123... maple-private-prod overlay swarm
def456... maple-public-prod overlay swarm
abc123... mapleopentech-private-prod overlay swarm
def456... mapleopentech-public-prod overlay swarm
```
**Why two networks?**
@ -380,7 +380,7 @@ def456... maple-public-prod overlay swarm
- **Public**: NGINX forwards requests to Backend (internet-facing)
- Backend joins BOTH networks to receive requests and access databases
**✅ Checkpoint:** Both `maple-private-prod` and `maple-public-prod` networks exist
**✅ Checkpoint:** Both `mapleopentech-private-prod` and `mapleopentech-public-prod` networks exist
---
@ -601,9 +601,9 @@ vi maplepress-stack.yml
version: '3.8'
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
maple-public-prod:
mapleopentech-public-prod:
external: true
secrets:
@ -625,8 +625,8 @@ services:
image: registry.digitalocean.com/ssp/maplepress_backend:prod
hostname: maplepress-backend
networks:
- maple-public-prod # Receives requests from Caddy
- maple-private-prod # Accesses databases
- mapleopentech-public-prod # Receives requests from Caddy
- mapleopentech-private-prod # Accesses databases
secrets:
- maplepress_jwt_secret
- maplepress_ip_encryption_key
@ -871,7 +871,7 @@ docker service logs maplepress_backend --tail 100
# 1. Can't reach databases
# - Verify Cassandra running: docker service ls | grep cassandra
# - Verify Redis running: docker service ls | grep redis
# - Check backend is on maple-private-prod network
# - Check backend is on mapleopentech-private-prod network
# 2. Secrets missing
docker secret ls
@ -1047,9 +1047,9 @@ docker service logs -f maplepress_backend
docker service ls
# Should show: cassandra_cassandra-1, cassandra_cassandra-2, cassandra_cassandra-3, redis_redis, meilisearch_meilisearch
# 2. Verify backend is on maple-private-prod network
# 2. Verify backend is on mapleopentech-private-prod network
docker service inspect maplepress_backend --format '{{range .Spec.TaskTemplate.Networks}}{{.Target}} {{end}}'
# Should include maple-private-prod
# Should include mapleopentech-private-prod
# 3. Test DNS resolution from backend container
ssh dockeradmin@<worker-6-public-ip>

34
cloud/infrastructure/production/setup/06_maplepress_caddy.md
View file

@ -4,7 +4,7 @@
**Time to Complete**: 20-30 minutes
**Prerequisites**:
- ✅ Completed guide **05_backend.md** (Backend deployed and running)
- ✅ Backend service accessible on `maple-public-prod` network
- ✅ Backend service accessible on `mapleopentech-public-prod` network
- ✅ Domain name pointing to worker-6 public IP
- ✅ Email address for Let's Encrypt SSL certificate notifications
@ -59,7 +59,7 @@ Backend (worker-6)
Databases (Cassandra, Redis, Meilisearch on other workers)
```
**Key concept:** Caddy and Backend are both on worker-6, connected via the `maple-public-prod` Docker overlay network. Caddy can reach Backend by the hostname `backend` - Docker's built-in DNS resolves this to the backend container's IP automatically.
**Key concept:** Caddy and Backend are both on worker-6, connected via the `mapleopentech-public-prod` Docker overlay network. Caddy can reach Backend by the hostname `backend` - Docker's built-in DNS resolves this to the backend container's IP automatically.
---
@ -328,9 +328,9 @@ vi maplepress-stack.yml
version: '3.8'
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
maple-public-prod:
mapleopentech-public-prod:
external: true
volumes:
@ -362,8 +362,8 @@ services:
image: registry.digitalocean.com/ssp/maplepress_backend:prod
hostname: maplepress-backend
networks:
- maple-public-prod
- maple-private-prod
- mapleopentech-public-prod
- mapleopentech-private-prod
secrets:
- maplepress_jwt_secret
- maplepress_ip_encryption_key
@ -431,7 +431,7 @@ services:
image: caddy:2-alpine
hostname: caddy
networks:
- maple-public-prod
- mapleopentech-public-prod
ports:
# Port 80 - HTTP (for Let's Encrypt challenges and HTTP→HTTPS redirect)
# Using mode: host to bind directly to worker-6's network interface
@ -488,7 +488,7 @@ Save: `Esc`, then `:wq`, then `Enter`
**Understanding the stack file:**
- **`maple-public-prod` network**: Shared network with backend
- **`mapleopentech-public-prod` network**: Shared network with backend
- Both Caddy and Backend are connected here
- Allows Caddy to reach Backend by hostname
- `external: true` means we created this network earlier (in 05_backend.md)
@ -542,9 +542,9 @@ Before deploying, let's make sure everything is ready:
docker service ls | grep backend
# Should show: maplepress_backend 1/1 registry.digitalocean.com/ssp/maplepress_backend:prod
# 2. Check maple-public-prod network exists
docker network ls | grep maple-public-prod
# Should show: maple-public-prod overlay swarm
# 2. Check mapleopentech-public-prod network exists
docker network ls | grep mapleopentech-public-prod
# Should show: mapleopentech-public-prod overlay swarm
# 3. Verify worker-6 has backend label
docker node inspect mapleopentech-swarm-worker-6-prod --format '{{.Spec.Labels}}'
@ -734,8 +734,8 @@ All should show `1/1` in REPLICAS column.
docker network ls | grep maple
# Should show:
# abc123... maple-private-prod overlay swarm
# def456... maple-public-prod overlay swarm
# abc123... mapleopentech-private-prod overlay swarm
# def456... mapleopentech-public-prod overlay swarm
```
### 5.3 Verify Service Connectivity
@ -978,8 +978,8 @@ docker service logs maplepress_backend-caddy --tail 100
# - Stop conflicting service
# 3. Network not found
docker network ls | grep maple-public-prod
# If missing, create it: docker network create --driver overlay --attachable maple-public-prod
docker network ls | grep mapleopentech-public-prod
# If missing, create it: docker network create --driver overlay --attachable mapleopentech-public-prod
```
### Problem: Deployment Fails with "only updates to Labels are allowed"
@ -1245,7 +1245,7 @@ docker service ls | grep backend
# 2. Are both on same network?
docker service inspect maplepress_backend --format '{{range .Spec.TaskTemplate.Networks}}{{.Target}} {{end}}'
docker service inspect maplepress_backend-caddy --format '{{range .Spec.TaskTemplate.Networks}}{{.Target}} {{end}}'
# Both should include: maple-public-prod
# Both should include: mapleopentech-public-prod
# 3. Test from Caddy container
ssh dockeradmin@<worker-6-public-ip>
@ -1478,7 +1478,7 @@ Internet (HTTPS)
Caddy (worker-6) - Automatic SSL, reverse proxy
↓
Backend (worker-6) - Your Go application
↓ maple-private-prod network
↓ mapleopentech-private-prod network
Cassandra (workers 1-3) - Database cluster
Redis (worker-4) - Cache
Meilisearch (worker-5) - Search engine

2
cloud/infrastructure/production/setup/07_maplepress_frontend.md
View file

@ -67,7 +67,7 @@ Internet (HTTPS)
- **Image**: Ubuntu 24.04 LTS x64
- **Size**: Basic shared CPU, 1 GB / 1 vCPU ($6/mo) - Frontend is lightweight
- **Hostname**: `mapleopentech-swarm-worker-7-prod`
- **VPC Network**: Select same VPC as your swarm (maple-vpc-prod)
- **VPC Network**: Select same VPC as your swarm (mapleopentech-vpc-prod)
- **SSH Keys**: Add your SSH key
4. Click **Create Droplet**
5. Wait 1-2 minutes for droplet to provision

32
cloud/infrastructure/production/setup/09_maplefile_backend.md
View file

@ -364,8 +364,8 @@ dig www.maplefile.ca +short
We need two overlay networks for our services:
1. **maple-private-prod** - Backend connects to databases (already exists from guides 02-04)
2. **maple-public-prod** - Caddy and Backend communicate (already exists from guide 06)
1. **mapleopentech-private-prod** - Backend connects to databases (already exists from guides 02-04)
2. **mapleopentech-public-prod** - Caddy and Backend communicate (already exists from guide 06)
**On manager:**
@ -376,15 +376,15 @@ ssh dockeradmin@143.110.210.162
docker network ls | grep maple
# Should show:
# maple-private-prod overlay swarm
# maple-public-prod overlay swarm
# mapleopentech-private-prod overlay swarm
# mapleopentech-public-prod overlay swarm
```
**If maple-public-prod doesn't exist, create it:**
**If mapleopentech-public-prod doesn't exist, create it:**
```bash
# Create public network
docker network create --driver overlay --attachable maple-public-prod
docker network create --driver overlay --attachable mapleopentech-public-prod
# Verify both exist
docker network ls | grep maple
@ -395,7 +395,7 @@ docker network ls | grep maple
- **Public**: Caddy forwards requests to Backend (internet-facing)
- Backend joins BOTH networks to receive requests and access databases
**✅ Checkpoint:** Both `maple-private-prod` and `maple-public-prod` networks exist
**✅ Checkpoint:** Both `mapleopentech-private-prod` and `mapleopentech-public-prod` networks exist
---
@ -623,9 +623,9 @@ vi ~/stacks/maplefile-stack.yml
version: '3.8'
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
maple-public-prod:
mapleopentech-public-prod:
external: true
secrets:
@ -645,8 +645,8 @@ services:
image: registry.digitalocean.com/ssp/maplefile-backend:prod
hostname: maplefile-backend
networks:
- maple-public-prod # Receives requests from Caddy
- maple-private-prod # Accesses databases
- mapleopentech-public-prod # Receives requests from Caddy
- mapleopentech-private-prod # Accesses databases
secrets:
- maplefile_jwt_secret
- redis_password
@ -935,7 +935,7 @@ docker service logs maplefile_backend --tail 100
# 1. Can't reach databases
# - Verify Cassandra running: docker service ls | grep cassandra
# - Verify Redis running: docker service ls | grep redis
# - Check backend is on maple-private-prod network
# - Check backend is on mapleopentech-private-prod network
# 2. Secrets missing
docker secret ls
@ -1027,9 +1027,9 @@ docker stack deploy -c ~/stacks/maplefile-stack.yml maplefile
docker service ls | grep cassandra
# Should show: cassandra_cassandra-1, cassandra_cassandra-2, cassandra_cassandra-3
# 2. Verify backend is on maple-private-prod network
# 2. Verify backend is on mapleopentech-private-prod network
docker service inspect maplefile_backend --format '{{range .Spec.TaskTemplate.Networks}}{{.Target}} {{end}}'
# Should include maple-private-prod
# Should include mapleopentech-private-prod
# 3. Test DNS resolution from backend container
ssh dockeradmin@143.110.212.253
@ -1189,14 +1189,14 @@ cd ~/stacks
## Network Architecture Notes
**Private Network (maple-private-prod):**
**Private Network (mapleopentech-private-prod):**
- Cassandra cluster (3 nodes) - Shared by ALL apps
- Redis - Shared by ALL apps (different DB numbers per app)
- Both MaplePress and MapleFile use same infrastructure
- MaplePress uses Cassandra keyspace: `maplepress`, Redis DB: 0
- MapleFile uses Cassandra keyspace: `maplefile`, Redis DB: 1
**Public Network (maple-public-prod):**
**Public Network (mapleopentech-public-prod):**
- Each app gets its own Caddy instance
- maplefile-backend-caddy (to be deployed in Part 2)
- maplepress-backend-caddy (already deployed)

14
cloud/infrastructure/production/setup/10_maplefile_caddy.md
View file

@ -4,7 +4,7 @@
**Time to Complete**: 20-30 minutes
**Prerequisites**:
- ✅ Completed guide **09_maplefile_backend.md** (Backend deployed and running)
- ✅ Backend service accessible on `maple-public-prod` network
- ✅ Backend service accessible on `mapleopentech-public-prod` network
- ✅ Domain name `maplefile.ca` pointing to worker-8 public IP
- ✅ Email address for Let's Encrypt SSL certificate notifications
@ -59,7 +59,7 @@ Backend (worker-8)
Databases (Cassandra, Redis on other workers)
```
**Key concept:** Caddy and Backend are both on worker-8, connected via the `maple-public-prod` Docker overlay network. Caddy can reach Backend by the hostname `maplefile-backend` - Docker's built-in DNS resolves this to the backend container's IP automatically.
**Key concept:** Caddy and Backend are both on worker-8, connected via the `mapleopentech-public-prod` Docker overlay network. Caddy can reach Backend by the hostname `maplefile-backend` - Docker's built-in DNS resolves this to the backend container's IP automatically.
---
@ -345,7 +345,7 @@ configs:
image: caddy:2.9.1-alpine
hostname: maplefile-caddy
networks:
- maple-public-prod
- mapleopentech-public-prod
ports:
# Port 80 - HTTP (for Let's Encrypt challenges and HTTP→HTTPS redirect)
# Using mode: host to bind directly to worker-8's network interface
@ -402,7 +402,7 @@ Save: `Esc`, then `:wq`, then `Enter`
**Understanding the stack file:**
- **`maple-public-prod` network**: Shared network with backend
- **`mapleopentech-public-prod` network**: Shared network with backend
- Both Caddy and Backend are connected here
- Allows Caddy to reach Backend by hostname
- `external: true` means we created this network earlier (in 09_maplefile_backend.md)
@ -431,7 +431,7 @@ Save: `Esc`, then `:wq`, then `Enter`
- **Placement constraint**:
- `node.labels.maplefile-backend == true` - Same as backend (worker-8)
- Caddy and Backend MUST be on the same node to share `maple-public-prod` network
- Caddy and Backend MUST be on the same node to share `mapleopentech-public-prod` network
- Docker overlay networks work best when services are colocated
### 3.2 Deploy Updated Stack
@ -679,12 +679,12 @@ docker service logs maplefile_backend-caddy --tail 50
# Should show: Running
```
2. **Backend not on maple-public-prod network**
2. **Backend not on mapleopentech-public-prod network**
```bash
# Check backend networks
docker service inspect maplefile_backend --format '{{json .Spec.TaskTemplate.Networks}}'
# Should include maple-public-prod
# Should include mapleopentech-public-prod
```
3. **Wrong hostname in Caddyfile**

2
cloud/infrastructure/production/setup/11_maplefile_frontend.md
View file

@ -75,7 +75,7 @@ Choose this approach if you want better isolation and independent scaling.
- **Image**: Ubuntu 24.04 LTS x64
- **Size**: Basic shared CPU, 1 GB / 1 vCPU ($6/mo) - Frontend is lightweight
- **Hostname**: `mapleopentech-swarm-worker-9-prod`
- **VPC Network**: Select same VPC as your swarm (maple-vpc-prod)
- **VPC Network**: Select same VPC as your swarm (mapleopentech-vpc-prod)
- **SSH Keys**: Add your SSH key
- **Tags**: Add tags like `production`, `maplefile`, `frontend`
4. Click **Create Droplet**

12
cloud/infrastructure/production/setup/README.md
View file

@ -27,9 +27,9 @@ Internet (HTTPS)
├─ getmaplepress.ca → Backend API (worker-6)
└─ getmaplepress.com → Frontend (worker-7)
↓
Backend Services (maple-public-prod + maple-private-prod)
Backend Services (mapleopentech-public-prod + mapleopentech-private-prod)
↓
Databases (maple-private-prod only)
Databases (mapleopentech-private-prod only)
├─ Cassandra: 3-node cluster (workers 2,3,4) - RF=3, QUORUM
├─ Redis: Single instance (worker-1/manager)
└─ Meilisearch: Single instance (worker-5)
@ -51,7 +51,7 @@ Internet (HTTPS)
- Command-line tools verification
**[00-network-architecture.md](00-network-architecture.md)** - Network design
- Network segmentation strategy (`maple-private-prod` vs `maple-public-prod`)
- Network segmentation strategy (`mapleopentech-private-prod` vs `mapleopentech-public-prod`)
- Security principles (defense in depth)
- Service communication patterns
- Firewall rules overview
@ -360,13 +360,13 @@ setup/
### Network Architecture
**`maple-private-prod` (overlay network):**
**`mapleopentech-private-prod` (overlay network):**
- All databases (Cassandra, Redis, Meilisearch)
- Backend services (access to databases)
- **No internet access** (security)
- Internal-only communication
**`maple-public-prod` (overlay network):**
**`mapleopentech-public-prod` (overlay network):**
- Caddy reverse proxies
- Backend services (receive HTTP requests)
- Ports 80/443 exposed to internet
@ -447,7 +447,7 @@ docker logs <container-id>
docker service inspect maplepress_backend
# Check network
docker network inspect maple-private-prod
docker network inspect mapleopentech-private-prod
# List configs
docker config ls

27
cloud/infrastructure/production/setup/templates/backend-stack.yml
View file

@ -1,9 +1,9 @@
version: '3.8'
version: "3.8"
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
maple-public-prod:
mapleopentech-public-prod:
external: true
secrets:
@ -24,8 +24,8 @@ services:
image: registry.digitalocean.com/ssp/maplepress_backend:latest
hostname: backend
networks:
- maple-public-prod # Receive requests from NGINX
- maple-private-prod # Access databases
- mapleopentech-public-prod # Receive requests from NGINX
- mapleopentech-private-prod # Access databases
secrets:
- maplepress_jwt_secret
- redis_password
@ -93,18 +93,27 @@ services:
resources:
limits:
memory: 1G
cpus: '1.0'
cpus: "1.0"
reservations:
memory: 512M
cpus: '0.5'
cpus: "0.5"
update_config:
parallelism: 1
delay: 10s
failure_action: rollback
order: start-first # Zero-downtime: start new before stopping old
order: start-first # Zero-downtime: start new before stopping old
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "--header=X-Tenant-ID: healthcheck", "http://localhost:8000/health"]
test:
[
"CMD",
"wget",
"--no-verbose",
"--tries=1",
"--spider",
"--header=X-Tenant-ID: healthcheck",
"http://localhost:8000/health",
]
interval: 30s
timeout: 5s
retries: 3

16
cloud/infrastructure/production/setup/templates/cassandra-stack.yml
View file

@ -1,7 +1,7 @@
version: '3.8'
version: "3.8"
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
volumes:
@ -14,9 +14,9 @@ services:
image: cassandra:5.0.4
hostname: cassandra-1
networks:
- maple-private-prod
- mapleopentech-private-prod
environment:
- CASSANDRA_CLUSTER_NAME=maple-prod-cluster
- CASSANDRA_CLUSTER_NAME=mapleopentech-prod-cluster
- CASSANDRA_DC=datacenter1
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
- CASSANDRA_SEEDS=cassandra-1,cassandra-2,cassandra-3
@ -44,9 +44,9 @@ services:
image: cassandra:5.0.4
hostname: cassandra-2
networks:
- maple-private-prod
- mapleopentech-private-prod
environment:
- CASSANDRA_CLUSTER_NAME=maple-prod-cluster
- CASSANDRA_CLUSTER_NAME=mapleopentech-prod-cluster
- CASSANDRA_DC=datacenter1
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
- CASSANDRA_SEEDS=cassandra-1,cassandra-2,cassandra-3
@ -74,9 +74,9 @@ services:
image: cassandra:5.0.4
hostname: cassandra-3
networks:
- maple-private-prod
- mapleopentech-private-prod
environment:
- CASSANDRA_CLUSTER_NAME=maple-prod-cluster
- CASSANDRA_CLUSTER_NAME=mapleopentech-prod-cluster
- CASSANDRA_DC=datacenter1
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
- CASSANDRA_SEEDS=cassandra-1,cassandra-2,cassandra-3

16
cloud/infrastructure/production/setup/templates/meilisearch-stack.yml
View file

@ -1,7 +1,7 @@
version: '3.8'
version: "3.8"
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
volumes:
@ -16,7 +16,7 @@ services:
image: getmeili/meilisearch:v1.5
hostname: meilisearch
networks:
- maple-private-prod
- mapleopentech-private-prod
volumes:
- meilisearch-data:/meili_data
secrets:
@ -49,7 +49,15 @@ services:
reservations:
memory: 768M
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:7700/health"]
test:
[
"CMD",
"wget",
"--no-verbose",
"--tries=1",
"--spider",
"http://localhost:7700/health",
]
interval: 30s
timeout: 10s
retries: 3

24
cloud/infrastructure/production/setup/templates/nginx-stack.yml
View file

@ -1,7 +1,7 @@
version: '3.8'
version: "3.8"
networks:
maple-public-prod:
mapleopentech-public-prod:
external: true
volumes:
@ -13,14 +13,14 @@ services:
image: nginx:alpine
hostname: nginx
networks:
- maple-public-prod
- mapleopentech-public-prod
ports:
- "80:80"
- "443:443"
volumes:
- nginx-ssl-certs:/etc/letsencrypt
- nginx-ssl-www:/var/www/certbot
- /var/run/docker.sock:/tmp/docker.sock:ro # For nginx-proxy
- /var/run/docker.sock:/tmp/docker.sock:ro # For nginx-proxy
configs:
- source: nginx_config
target: /etc/nginx/nginx.conf
@ -30,7 +30,7 @@ services:
replicas: 1
placement:
constraints:
- node.labels.backend == true # Same node as backend
- node.labels.backend == true # Same node as backend
restart_policy:
condition: on-failure
delay: 5s
@ -38,12 +38,20 @@ services:
resources:
limits:
memory: 256M
cpus: '0.5'
cpus: "0.5"
reservations:
memory: 128M
cpus: '0.25'
cpus: "0.25"
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:80/health"]
test:
[
"CMD",
"wget",
"--no-verbose",
"--tries=1",
"--spider",
"http://localhost:80/health",
]
interval: 30s
timeout: 5s
retries: 3

14
cloud/infrastructure/production/setup/templates/redis-stack.yml
View file

@ -1,7 +1,7 @@
version: '3.8'
version: "3.8"
networks:
maple-private-prod:
mapleopentech-private-prod:
external: true
volumes:
@ -16,7 +16,7 @@ services:
image: redis:7-alpine
hostname: redis
networks:
- maple-private-prod
- mapleopentech-private-prod
volumes:
- redis-data:/data
secrets:
@ -66,7 +66,13 @@ services:
reservations:
memory: 512M
healthcheck:
test: ["CMD", "sh", "-c", "redis-cli -a $$(cat /run/secrets/redis_password) ping | grep PONG"]
test:
[
"CMD",
"sh",
"-c",
"redis-cli -a $$(cat /run/secrets/redis_password) ping | grep PONG",
]
interval: 10s
timeout: 3s
retries: 3

2
cloud/infrastructure/production/setup/templates/redis.prod.conf
View file

@ -11,7 +11,7 @@ bind 0.0.0.0
port 6379
# Protected mode disabled (we rely on Docker network isolation)
# Only containers on maple-prod overlay network can access
# Only containers on mapleopentech-prod overlay network can access
protected-mode no
# ==============================================================================