Initial commit: Open sourcing all of the Maple Open Technologies code.
This commit is contained in:
commit
755d54a99d
2010 changed files with 448675 additions and 0 deletions
148
cloud/infrastructure/production/operations/BACKEND_ACCESS.md
Normal file
148
cloud/infrastructure/production/operations/BACKEND_ACCESS.md
Normal file
|
|
@ -0,0 +1,148 @@
|
|||
# Backend Access & Database Operations
|
||||
|
||||
## Access Backend Container
|
||||
|
||||
```bash
|
||||
# Find which node runs the backend
|
||||
ssh dockeradmin@<manager-ip>
|
||||
docker service ps maplefile_backend --filter "desired-state=running"
|
||||
# Note the NODE column
|
||||
|
||||
# SSH to that worker
|
||||
ssh dockeradmin@<worker-ip>
|
||||
|
||||
# Get container ID
|
||||
export BACKEND_CONTAINER=$(docker ps --filter "name=maplefile.*backend" -q | head -1)
|
||||
|
||||
# Open shell
|
||||
docker exec -it $BACKEND_CONTAINER sh
|
||||
|
||||
# Or run single command
|
||||
docker exec $BACKEND_CONTAINER ./maplefile-backend --help
|
||||
```
|
||||
|
||||
## View Logs
|
||||
|
||||
```bash
|
||||
# Follow logs
|
||||
docker logs -f $BACKEND_CONTAINER
|
||||
|
||||
# Last 100 lines
|
||||
docker logs --tail 100 $BACKEND_CONTAINER
|
||||
|
||||
# Search for errors
|
||||
docker logs $BACKEND_CONTAINER 2>&1 | grep -i error
|
||||
```
|
||||
|
||||
## Database Operations
|
||||
|
||||
### Run Migrations (Safe)
|
||||
|
||||
```bash
|
||||
docker exec $BACKEND_CONTAINER ./maplefile-backend migrate up
|
||||
```
|
||||
|
||||
Auto-runs on backend startup when `DATABASE_AUTO_MIGRATE=true` (default in stack file).
|
||||
|
||||
### Rollback Last Migration (Destructive)
|
||||
|
||||
```bash
|
||||
docker exec $BACKEND_CONTAINER ./maplefile-backend migrate down
|
||||
```
|
||||
|
||||
Only rolls back 1 migration. Run multiple times for multiple rollbacks.
|
||||
|
||||
### Reset Database (Full Wipe)
|
||||
|
||||
```bash
|
||||
# 1. SSH to any Cassandra node (any of the 3 nodes works)
|
||||
ssh dockeradmin@<cassandra-node-ip>
|
||||
|
||||
# 2. Find the Cassandra container ID
|
||||
export CASSANDRA_CONTAINER=$(docker ps --filter "name=cassandra" -q | head -1)
|
||||
|
||||
# 3. Drop keyspace (DELETES ALL DATA - propagates to all 3 nodes)
|
||||
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "DROP KEYSPACE IF EXISTS maplefile;"
|
||||
|
||||
# 4. Wait for schema to propagate across cluster
|
||||
sleep 5
|
||||
|
||||
# 5. Recreate keyspace (propagates to all 3 nodes)
|
||||
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "
|
||||
CREATE KEYSPACE IF NOT EXISTS maplefile
|
||||
WITH replication = {
|
||||
'class': 'SimpleStrategy',
|
||||
'replication_factor': 3
|
||||
};"
|
||||
|
||||
# 6. Wait for schema agreement across cluster
|
||||
sleep 5
|
||||
|
||||
# 7. Verify keyspace exists
|
||||
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "DESCRIBE KEYSPACE maplefile;"
|
||||
|
||||
# 8. Restart backend to run migrations
|
||||
# You must pull the new image on the worker node first
|
||||
# Find which worker runs the service:
|
||||
ssh dockeradmin@<manager-ip>
|
||||
docker service ps maplefile_backend
|
||||
# Note the worker node name
|
||||
|
||||
# Pull image on the worker:
|
||||
ssh dockeradmin@<worker-ip>
|
||||
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
exit
|
||||
|
||||
# Force restart on manager:
|
||||
ssh dockeradmin@<manager-ip>
|
||||
docker service update --force maplefile_backend
|
||||
|
||||
# Verify new version is running:
|
||||
docker service logs maplefile_backend --tail 50
|
||||
# Look for: 📝 Git Commit: <commit-sha>
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Container Not Found
|
||||
|
||||
```bash
|
||||
# Check service status
|
||||
docker service ps maplefile_backend
|
||||
|
||||
# List all backend containers
|
||||
docker ps | grep backend
|
||||
```
|
||||
|
||||
### Wrong Container (MaplePress vs MapleFile)
|
||||
|
||||
```bash
|
||||
# Verify you have MapleFile (not MaplePress)
|
||||
docker ps | grep $BACKEND_CONTAINER
|
||||
# Should show "maplefile-backend" in image name
|
||||
```
|
||||
|
||||
### Migration Fails
|
||||
|
||||
```bash
|
||||
# Check environment (from worker node)
|
||||
docker exec $BACKEND_CONTAINER env | grep DATABASE
|
||||
|
||||
# Check Cassandra connectivity
|
||||
docker exec $BACKEND_CONTAINER nc -zv cassandra-1 9042
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Environment variables are in `~/stacks/maplefile-stack.yml` on manager node, not `.env` files.
|
||||
|
||||
To change config:
|
||||
1. Edit `~/stacks/maplefile-stack.yml`
|
||||
2. Pull new image on worker: `ssh dockeradmin@<worker-ip> && docker pull registry.digitalocean.com/ssp/maplefile-backend:prod && exit`
|
||||
3. Force restart on manager: `docker service update --force maplefile_backend`
|
||||
|
||||
**Important**: Worker nodes cache images locally. You MUST pull the new image on the worker node before restarting the service. The `--resolve-image always` and `--with-registry-auth` flags do NOT reliably force worker nodes to pull new images.
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: November 2025
|
||||
196
cloud/infrastructure/production/operations/BACKEND_UPDATES.md
Normal file
196
cloud/infrastructure/production/operations/BACKEND_UPDATES.md
Normal file
|
|
@ -0,0 +1,196 @@
|
|||
# Docker Image Updates & Deployment
|
||||
|
||||
**Quick Reference for MapleFile & MaplePress Backend**
|
||||
|
||||
## Images
|
||||
|
||||
- MapleFile: `registry.digitalocean.com/ssp/maplefile-backend:prod`
|
||||
- MaplePress: `registry.digitalocean.com/ssp/maplepress-backend:prod`
|
||||
|
||||
## Build & Push
|
||||
|
||||
```bash
|
||||
cd ~/go/src/codeberg.org/mapleopentech/monorepo/cloud/maplefile-backend
|
||||
task deploy
|
||||
# Note the Image ID and git commit from output
|
||||
docker images registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
```
|
||||
|
||||
## Deploy to Production
|
||||
|
||||
**CRITICAL**: Docker Swarm caches images. You MUST verify Image IDs match across all nodes.
|
||||
|
||||
### Step 1: Note Your Local Image ID
|
||||
|
||||
```bash
|
||||
docker images registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
# Example: IMAGE ID = 74b2fafb1f69
|
||||
```
|
||||
|
||||
### Step 2: Find Worker Node & Pull Images
|
||||
|
||||
```bash
|
||||
# SSH to manager
|
||||
ssh dockeradmin@<MANAGER_IP>
|
||||
|
||||
# Find which worker runs the service
|
||||
docker service ps maplefile_backend
|
||||
# Note: NODE column (e.g., mapleopentech-swarm-worker-8-prod)
|
||||
|
||||
# Pull on manager
|
||||
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
|
||||
# Pull on worker
|
||||
ssh dockeradmin@<WORKER_NODE>
|
||||
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
exit
|
||||
```
|
||||
|
||||
### Step 3: Verify Image IDs Match
|
||||
|
||||
```bash
|
||||
# On manager
|
||||
docker images registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
|
||||
# On worker
|
||||
ssh dockeradmin@<WORKER_NODE>
|
||||
docker images registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
exit
|
||||
|
||||
# ALL THREE (local, manager, worker) must show SAME Image ID
|
||||
```
|
||||
|
||||
### Step 4: Remove & Recreate Service
|
||||
|
||||
```bash
|
||||
# On manager - remove service
|
||||
docker service rm maplefile_backend
|
||||
|
||||
# Redeploy stack
|
||||
cd ~/stacks
|
||||
docker stack deploy -c maplefile-stack.yml maplefile
|
||||
```
|
||||
|
||||
### Step 5: Verify Deployment
|
||||
|
||||
```bash
|
||||
docker service logs maplefile_backend --tail 50
|
||||
|
||||
# Confirm these match your build:
|
||||
# 🚀 Starting MapleFile Backend v0.1.0
|
||||
# 📝 Git Commit: <your-commit-sha>
|
||||
# 🕐 Build Time: <your-build-timestamp>
|
||||
```
|
||||
|
||||
## For MaplePress
|
||||
|
||||
Same process, replace `maplefile` with `maplepress`:
|
||||
|
||||
```bash
|
||||
docker service ps maplepress_backend
|
||||
# Pull on both nodes
|
||||
docker service rm maplepress_backend
|
||||
docker stack deploy -c maplepress-stack.yml maplepress
|
||||
```
|
||||
|
||||
## Why Remove & Recreate?
|
||||
|
||||
Docker Swarm's `docker service update --force` does NOT reliably use new images even after pulling. The `--resolve-image always` and `--with-registry-auth` flags also fail with mutable `:prod` tags.
|
||||
|
||||
**Only remove & recreate guarantees the new image is used.**
|
||||
|
||||
## Rollback
|
||||
|
||||
### Quick Rollback
|
||||
|
||||
```bash
|
||||
# Automatic rollback to previous version
|
||||
docker service rollback maplefile_backend
|
||||
```
|
||||
|
||||
### Rollback to Specific Version
|
||||
|
||||
```bash
|
||||
# Find previous image digest
|
||||
docker service ps maplefile_backend --no-trunc
|
||||
|
||||
# Rollback to specific digest
|
||||
docker service update --image registry.digitalocean.com/ssp/maplefile-backend:prod@sha256:def456... maplefile_backend
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Health Check Failures
|
||||
|
||||
```bash
|
||||
# Check logs
|
||||
docker service logs maplefile_backend --tail 100
|
||||
|
||||
# Rollback if needed
|
||||
docker service rollback maplefile_backend
|
||||
```
|
||||
|
||||
### Image Pull Authentication Error
|
||||
|
||||
```bash
|
||||
# Re-authenticate
|
||||
doctl registry login
|
||||
|
||||
# Retry
|
||||
docker service update --image registry.digitalocean.com/ssp/maplefile-backend:prod maplefile_backend
|
||||
```
|
||||
|
||||
### Service Stuck Starting
|
||||
|
||||
```bash
|
||||
# Common causes: database migrations failing, missing env vars, health check issues
|
||||
# Check logs
|
||||
docker service logs maplefile_backend --tail 50
|
||||
|
||||
# Rollback if urgent
|
||||
docker service rollback maplefile_backend
|
||||
```
|
||||
|
||||
## Standard Deployment Workflow
|
||||
|
||||
```bash
|
||||
# 1. Local: Build & push (note the git commit and Image ID)
|
||||
cd ~/go/src/codeberg.org/mapleopentech/monorepo/cloud/maplefile-backend
|
||||
task deploy
|
||||
# Example output: "Deployed version d90b6e2b - use this to verify on production"
|
||||
|
||||
# Note the local Image ID for verification
|
||||
docker images registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
# Example: IMAGE ID = 74b2fafb1f69
|
||||
|
||||
# 2. Find which worker is running the service
|
||||
ssh dockeradmin@<MANAGER_IP>
|
||||
docker service ps maplefile_backend
|
||||
# Note the worker node (e.g., mapleopentech-swarm-worker-8-prod)
|
||||
|
||||
# 3. Pull the new image on the MANAGER node
|
||||
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
|
||||
# 4. Pull the new image on the WORKER node
|
||||
ssh dockeradmin@<WORKER_NODE>
|
||||
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod
|
||||
exit
|
||||
|
||||
# 5. Force restart on manager
|
||||
ssh dockeradmin@<MANAGER_IP>
|
||||
docker service update --force maplefile_backend
|
||||
|
||||
# 6. Verify git commit matches what you deployed
|
||||
docker service logs maplefile_backend --tail 50
|
||||
# Look for: 📝 Git Commit: d90b6e2b...
|
||||
```
|
||||
|
||||
**Key points**:
|
||||
- You MUST pull the image on **BOTH manager and worker nodes**
|
||||
- Use `docker images` to verify Image ID matches your local build
|
||||
- Use `docker service update --force` to restart with the new image
|
||||
- Check startup logs for Git Commit to verify correct version is running
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: November 2025
|
||||
15
cloud/infrastructure/production/operations/DEBUGGING.md
Normal file
15
cloud/infrastructure/production/operations/DEBUGGING.md
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
To see console log of our backend:
|
||||
|
||||
Log in the specific worker node.
|
||||
|
||||
Afterwords run in the conosle"
|
||||
|
||||
```shell
|
||||
docker ps | grep backend
|
||||
```
|
||||
|
||||
and then:
|
||||
|
||||
```shell
|
||||
docker logs -f aa1b2c65eba7
|
||||
```
|
||||
1004
cloud/infrastructure/production/operations/ENVIRONMENT_VARIABLES.md
Normal file
1004
cloud/infrastructure/production/operations/ENVIRONMENT_VARIABLES.md
Normal file
File diff suppressed because it is too large
Load diff
124
cloud/infrastructure/production/operations/FRONTEND_UPDATES.md
Normal file
124
cloud/infrastructure/production/operations/FRONTEND_UPDATES.md
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
# Frontend Updates & Deployment
|
||||
|
||||
**Quick Reference for MapleFile Frontend**
|
||||
|
||||
## Overview
|
||||
|
||||
The frontend runs on Worker Node 9 as a static site built with Vite/React. Updates are deployed by pulling the latest code and rebuilding.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- SSH access to worker-9 as `dockeradmin`
|
||||
- Node.js and npm installed on the server
|
||||
|
||||
## Quick Deploy
|
||||
|
||||
```bash
|
||||
# SSH to worker-9 and run deploy script
|
||||
ssh dockeradmin@<WORKER_9_IP>
|
||||
~/deploy-frontend.sh
|
||||
```
|
||||
|
||||
## Manual Deploy
|
||||
|
||||
```bash
|
||||
# 1. SSH to worker-9
|
||||
ssh dockeradmin@<WORKER_9_IP>
|
||||
|
||||
# 2. Navigate to monorepo
|
||||
cd /var/www/monorepo
|
||||
|
||||
# 3. Pull latest changes (includes .env.production from git)
|
||||
git pull origin main
|
||||
|
||||
# 4. Navigate to frontend
|
||||
cd web/maplefile-frontend
|
||||
|
||||
# 5. Install dependencies (if package.json changed)
|
||||
npm install
|
||||
|
||||
# 6. Build production bundle
|
||||
npm run build
|
||||
```
|
||||
|
||||
## Verify Deployment
|
||||
|
||||
```bash
|
||||
# Check build output exists
|
||||
ls -la /var/www/monorepo/web/maplefile-frontend/dist/
|
||||
|
||||
# Check build timestamp
|
||||
stat /var/www/monorepo/web/maplefile-frontend/dist/index.html
|
||||
```
|
||||
|
||||
## Rollback
|
||||
|
||||
```bash
|
||||
# SSH to worker-9
|
||||
ssh dockeradmin@<WORKER_9_IP>
|
||||
|
||||
# Navigate to monorepo
|
||||
cd /var/www/monorepo
|
||||
|
||||
# Reset to previous commit
|
||||
git log --oneline -10 # Find the commit to rollback to
|
||||
git checkout <COMMIT_SHA>
|
||||
|
||||
# Rebuild
|
||||
cd web/maplefile-frontend
|
||||
npm install
|
||||
npm run build
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Build Fails
|
||||
|
||||
```bash
|
||||
# Clear node_modules and rebuild
|
||||
cd /var/www/monorepo/web/maplefile-frontend
|
||||
rm -rf node_modules
|
||||
npm install
|
||||
npm run build
|
||||
```
|
||||
|
||||
### Check Node.js Version
|
||||
|
||||
```bash
|
||||
node --version
|
||||
npm --version
|
||||
|
||||
# If outdated, update Node.js
|
||||
```
|
||||
|
||||
### Permission Issues
|
||||
|
||||
```bash
|
||||
# Ensure correct ownership
|
||||
sudo chown -R dockeradmin:dockeradmin /var/www/monorepo
|
||||
```
|
||||
|
||||
## Standard Deployment Workflow
|
||||
|
||||
```bash
|
||||
# 1. Local: Commit and push your changes
|
||||
cd ~/go/src/codeberg.org/mapleopentech/monorepo/web/maplefile-frontend
|
||||
git add .
|
||||
git commit -m "feat: your changes"
|
||||
git push origin main
|
||||
|
||||
# 2. Deploy to production
|
||||
ssh dockeradmin@<WORKER_9_IP>
|
||||
cd /var/www/monorepo
|
||||
git pull origin main
|
||||
cd web/maplefile-frontend
|
||||
npm install
|
||||
npm run build
|
||||
|
||||
# 3. Verify by visiting the site
|
||||
# https://maplefile.app (or your domain)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: November 2025
|
||||
1097
cloud/infrastructure/production/operations/HORIZONTAL_SCALING.md
Normal file
1097
cloud/infrastructure/production/operations/HORIZONTAL_SCALING.md
Normal file
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue