Initial commit: Open sourcing all of the Maple Open Technologies code.
This commit is contained in:
commit
755d54a99d
2010 changed files with 448675 additions and 0 deletions
294
cloud/infrastructure/production/setup/00-network-architecture.md
Normal file
294
cloud/infrastructure/production/setup/00-network-architecture.md
Normal file
|
|
@ -0,0 +1,294 @@
|
|||
# Network Architecture Overview
|
||||
|
||||
This document explains the network strategy for Maple Open Technologies production infrastructure.
|
||||
|
||||
**See Also**: `00-multi-app-architecture.md` for application naming conventions and multi-app strategy.
|
||||
|
||||
## Network Segmentation Strategy
|
||||
|
||||
We use a **multi-network architecture** following industry best practices for security and isolation. This infrastructure supports **multiple independent applications** (MaplePress, MapleFile) sharing common infrastructure.
|
||||
|
||||
### Network Topology
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Docker Swarm Cluster │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────────┐ │
|
||||
│ │ maple-private-prod (Overlay Network) │ │
|
||||
│ │ No Internet Access | Internal Services Only │ │
|
||||
│ │ SHARED by ALL applications │ │
|
||||
│ ├────────────────────────────────────────────────────────────┤ │
|
||||
│ │ Infrastructure Services: │ │
|
||||
│ │ ├── Cassandra (3 nodes) - Shared database cluster │ │
|
||||
│ │ ├── Redis - Shared cache │ │
|
||||
│ │ └── Meilisearch - Shared search │ │
|
||||
│ │ │ │
|
||||
│ │ Application Backends (Join BOTH Networks): │ │
|
||||
│ │ ├── maplepress-backend:8000 │ │
|
||||
│ │ ├── maplefile-backend:8000 (future) │ │
|
||||
│ │ └── mapleopentech-backend:8000 (future) │ │
|
||||
│ └────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────────┐ │
|
||||
│ │ maple-public-prod (Overlay Network) │ │
|
||||
│ │ Internet-Facing | Public Services │ │
|
||||
│ ├────────────────────────────────────────────────────────────┤ │
|
||||
│ │ Reverse Proxies (Caddy - ports 80/443): │ │
|
||||
│ │ ├── maplepress-backend-caddy (getmaplepress.ca) │ │
|
||||
│ │ ├── maplepress-frontend-caddy (getmaplepress.com) │ │
|
||||
│ │ ├── maplefile-backend-caddy (maplefile.ca) │ │
|
||||
│ │ ├── maplefile-frontend-caddy (maplefile.com) │ │
|
||||
│ │ └── ... (future apps) │ │
|
||||
│ │ │ │
|
||||
│ │ Application Backends (Join BOTH Networks): │ │
|
||||
│ │ ├── maplepress-backend:8000 │ │
|
||||
│ │ ├── maplefile-backend:8000 (future) │ │
|
||||
│ │ └── mapleopentech-backend:8000 (future) │ │
|
||||
│ └────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Note: Application backends join BOTH networks: │
|
||||
│ - Receive requests from Caddy on maple-public-prod │
|
||||
│ - Access databases/cache on maple-private-prod │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Networks Explained
|
||||
|
||||
### 1. `maple-private-prod` (Current)
|
||||
|
||||
**Purpose**: Backend services that should NEVER be exposed to the internet.
|
||||
|
||||
**Characteristics:**
|
||||
- Overlay network (Docker Swarm managed)
|
||||
- No ingress ports exposed
|
||||
- No public IP access
|
||||
- Service-to-service communication only
|
||||
|
||||
**Services:**
|
||||
- Cassandra cluster (3 nodes - shared database) - databases never touch internet
|
||||
- Redis (shared cache layer)
|
||||
- Meilisearch (shared search engine)
|
||||
- **All application backends** (maplepress-backend, maplefile-backend, mapleopentech-backend)
|
||||
|
||||
**Security Benefits:**
|
||||
- Attack surface minimization
|
||||
- No direct internet access to databases
|
||||
- Compliance with data protection regulations (PCI-DSS, HIPAA, SOC2)
|
||||
- Defense in depth architecture
|
||||
|
||||
**Service Discovery:**
|
||||
```bash
|
||||
# Services can reach each other by hostname
|
||||
redis:6379
|
||||
cassandra-1:9042
|
||||
cassandra-2:9042
|
||||
cassandra-3:9042
|
||||
```
|
||||
|
||||
### 2. `maple-public-prod` (Current - In Use)
|
||||
|
||||
**Purpose**: Internet-facing services that handle external traffic.
|
||||
|
||||
**Characteristics:**
|
||||
- Overlay network with ingress
|
||||
- Ports 80/443 exposed to internet
|
||||
- TLS/SSL termination via Caddy
|
||||
- Automatic Let's Encrypt certificates
|
||||
- Rate limiting and security headers
|
||||
|
||||
**Services:**
|
||||
- **Caddy reverse proxies** (one per app component):
|
||||
- `maplepress-backend-caddy` → serves `getmaplepress.ca` → proxies to `maplepress-backend:8000`
|
||||
- `maplepress-frontend-caddy` → serves `getmaplepress.com` → static React files
|
||||
- `maplefile-backend-caddy` (future) → serves `maplefile.ca` → proxies to `maplefile-backend:8000`
|
||||
- `maplefile-frontend-caddy` (future) → serves `maplefile.com` → static React files
|
||||
- **All application backends** (join both networks):
|
||||
- `maplepress-backend`
|
||||
- `maplefile-backend` (future)
|
||||
- `mapleopentech-backend` (future)
|
||||
|
||||
**Routing Flow:**
|
||||
```
|
||||
Internet → Caddy Reverse Proxy (maple-public-prod)
|
||||
→ Application Backend (maple-public-prod + maple-private-prod)
|
||||
→ Databases/Cache (maple-private-prod only)
|
||||
|
||||
Example (MaplePress):
|
||||
https://getmaplepress.ca → maplepress-backend-caddy
|
||||
→ maplepress-backend:8000
|
||||
→ cassandra/redis/meilisearch
|
||||
```
|
||||
|
||||
## Why This Architecture?
|
||||
|
||||
### Industry Standard
|
||||
|
||||
This pattern is used by:
|
||||
- **Netflix**: `backend-network` + `edge-network`
|
||||
- **Spotify**: `data-plane` + `control-plane`
|
||||
- **AWS**: VPC with `private-subnet` + `public-subnet`
|
||||
- **Google Cloud**: VPC with internal + external networks
|
||||
|
||||
### Security Benefits
|
||||
|
||||
1. **Defense in Depth**: Multiple security layers
|
||||
2. **Least Privilege**: Services only access what they need
|
||||
3. **Attack Surface Reduction**: Databases never exposed to internet
|
||||
4. **Network Segmentation**: Compliance requirement for SOC2, PCI-DSS
|
||||
5. **Blast Radius Containment**: Breach of public network doesn't compromise data layer
|
||||
|
||||
### Operational Benefits
|
||||
|
||||
1. **Clear Boundaries**: Easy to understand what's exposed
|
||||
2. **Independent Scaling**: Scale public/private networks separately
|
||||
3. **Flexible Firewall Rules**: Different rules for different networks
|
||||
4. **Service Discovery**: DNS-based discovery within each network
|
||||
5. **Testing**: Can test private services without public exposure
|
||||
|
||||
## Network Creation
|
||||
|
||||
### Current Setup
|
||||
|
||||
Both networks are created and in use:
|
||||
|
||||
```bash
|
||||
# Create private network (done in 02_cassandra.md - shared by ALL apps)
|
||||
docker network create \
|
||||
--driver overlay \
|
||||
--attachable \
|
||||
maple-private-prod
|
||||
|
||||
# Create public network (done in 06_caddy.md - used by reverse proxies)
|
||||
docker network create \
|
||||
--driver overlay \
|
||||
--attachable \
|
||||
maple-public-prod
|
||||
|
||||
# Verify both exist
|
||||
docker network ls | grep maple
|
||||
# Should show:
|
||||
# maple-private-prod
|
||||
# maple-public-prod
|
||||
```
|
||||
|
||||
### Multi-App Pattern
|
||||
|
||||
- **All application backends** join BOTH networks
|
||||
- **Each app** gets its own Caddy reverse proxy instances
|
||||
- **Infrastructure services** (Cassandra, Redis, Meilisearch) only on private network
|
||||
- **Shared efficiently**: 5 infrastructure workers serve unlimited apps
|
||||
|
||||
## Service Connection Examples
|
||||
|
||||
### Go Backend Connecting to Services
|
||||
|
||||
**On `maple-private-prod` network:**
|
||||
|
||||
```go
|
||||
// Redis connection
|
||||
redisClient := redis.NewClient(&redis.Options{
|
||||
Addr: "redis:6379", // Resolves via Docker DNS
|
||||
Password: os.Getenv("REDIS_PASSWORD"),
|
||||
})
|
||||
|
||||
// Cassandra connection
|
||||
cluster := gocql.NewCluster("cassandra-1", "cassandra-2", "cassandra-3")
|
||||
cluster.Port = 9042
|
||||
```
|
||||
|
||||
**Docker Stack File for Backend:**
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
backend:
|
||||
image: your-backend:latest
|
||||
networks:
|
||||
- maple-private-prod # Access to databases
|
||||
- maple-public-prod # Receive HTTP requests (when deployed)
|
||||
environment:
|
||||
- REDIS_HOST=redis
|
||||
- CASSANDRA_HOSTS=cassandra-1,cassandra-2,cassandra-3
|
||||
|
||||
networks:
|
||||
maple-private-prod:
|
||||
external: true
|
||||
maple-public-prod:
|
||||
external: true
|
||||
```
|
||||
|
||||
## Firewall Rules
|
||||
|
||||
### Private Network
|
||||
|
||||
```bash
|
||||
# On worker nodes
|
||||
# Only allow traffic from other swarm nodes (10.116.0.0/16)
|
||||
sudo ufw allow from 10.116.0.0/16 to any port 2377 proto tcp # Swarm
|
||||
sudo ufw allow from 10.116.0.0/16 to any port 7946 # Gossip
|
||||
sudo ufw allow from 10.116.0.0/16 to any port 4789 proto udp # Overlay
|
||||
sudo ufw allow from 10.116.0.0/16 to any port 6379 proto tcp # Redis
|
||||
sudo ufw allow from 10.116.0.0/16 to any port 9042 proto tcp # Cassandra
|
||||
```
|
||||
|
||||
### Public Network (Caddy Nodes)
|
||||
|
||||
```bash
|
||||
# On workers running Caddy (worker-6, worker-7, worker-8, worker-9, etc.)
|
||||
sudo ufw allow 80/tcp # HTTP (Let's Encrypt challenge + redirect to HTTPS)
|
||||
sudo ufw allow 443/tcp # HTTPS (TLS/SSL traffic)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Check Which Networks a Service Uses
|
||||
|
||||
```bash
|
||||
# Inspect service networks
|
||||
docker service inspect your_service --format '{{.Spec.TaskTemplate.Networks}}'
|
||||
|
||||
# Should show network IDs
|
||||
# Compare with: docker network ls
|
||||
```
|
||||
|
||||
### Test Connectivity Between Networks
|
||||
|
||||
```bash
|
||||
# From a container on maple-private-prod
|
||||
docker exec -it <container> ping redis
|
||||
docker exec -it <container> nc -zv cassandra-1 9042
|
||||
|
||||
# Should work if on same network
|
||||
```
|
||||
|
||||
### View All Services on a Network
|
||||
|
||||
```bash
|
||||
docker network inspect maple-private-prod --format '{{range .Containers}}{{.Name}} {{end}}'
|
||||
```
|
||||
|
||||
## Migration Path
|
||||
|
||||
### Current Status
|
||||
- ✅ `maple-private-prod` created
|
||||
- ✅ Cassandra on `maple-private-prod`
|
||||
- ✅ Redis on `maple-private-prod`
|
||||
- ⏳ Backend deployment (next)
|
||||
- ⏳ Public network + NGINX (future)
|
||||
|
||||
### When to Create `maple-public-prod`
|
||||
|
||||
Create the public network when you're ready to:
|
||||
1. Deploy NGINX reverse proxy
|
||||
2. Set up SSL/TLS certificates
|
||||
3. Expose your application to the internet
|
||||
|
||||
Until then, all services run on the private network only.
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: November 3, 2025
|
||||
**Status**: Active Architecture
|
||||
**Maintained By**: Infrastructure Team
|
||||
Loading…
Add table
Add a link
Reference in a new issue