monorepo/cloud/infrastructure/production/operations/BACKEND_UPDATES.md

Docker Image Updates & Deployment

Quick Reference for MapleFile & MaplePress Backend

Images

• MapleFile: registry.digitalocean.com/ssp/maplefile-backend:prod
• MaplePress: registry.digitalocean.com/ssp/maplepress-backend:prod

Build & Push

cd ~/go/src/codeberg.org/mapleopentech/monorepo/cloud/maplefile-backend
task deploy
# Note the Image ID and git commit from output
docker images registry.digitalocean.com/ssp/maplefile-backend:prod

Deploy to Production

CRITICAL: Docker Swarm caches images. You MUST verify Image IDs match across all nodes.

Step 1: Note Your Local Image ID

docker images registry.digitalocean.com/ssp/maplefile-backend:prod
# Example: IMAGE ID = 74b2fafb1f69

Step 2: Find Worker Node & Pull Images

# SSH to manager
ssh dockeradmin@<MANAGER_IP>

# Find which worker runs the service
docker service ps maplefile_backend
# Note: NODE column (e.g., mapleopentech-swarm-worker-8-prod)

# Pull on manager
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod

# Pull on worker
ssh dockeradmin@<WORKER_NODE>
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod
exit

Step 3: Verify Image IDs Match

# On manager
docker images registry.digitalocean.com/ssp/maplefile-backend:prod

# On worker
ssh dockeradmin@<WORKER_NODE>
docker images registry.digitalocean.com/ssp/maplefile-backend:prod
exit

# ALL THREE (local, manager, worker) must show SAME Image ID

Step 4: Remove & Recreate Service

# On manager - remove service
docker service rm maplefile_backend

# Redeploy stack
cd ~/stacks
docker stack deploy -c maplefile-stack.yml maplefile

Step 5: Verify Deployment

docker service logs maplefile_backend --tail 50

# Confirm these match your build:
# 🚀 Starting MapleFile Backend v0.1.0
# 📝 Git Commit: <your-commit-sha>
# 🕐 Build Time: <your-build-timestamp>

For MaplePress

Same process, replace maplefile with maplepress:

docker service ps maplepress_backend
# Pull on both nodes
docker service rm maplepress_backend
docker stack deploy -c maplepress-stack.yml maplepress

Why Remove & Recreate?

Docker Swarm's docker service update --force does NOT reliably use new images even after pulling. The --resolve-image always and --with-registry-auth flags also fail with mutable :prod tags.

Only remove & recreate guarantees the new image is used.

Rollback

Quick Rollback

# Automatic rollback to previous version
docker service rollback maplefile_backend

Rollback to Specific Version

# Find previous image digest
docker service ps maplefile_backend --no-trunc

# Rollback to specific digest
docker service update --image registry.digitalocean.com/ssp/maplefile-backend:prod@sha256:def456... maplefile_backend

Troubleshooting

Health Check Failures

# Check logs
docker service logs maplefile_backend --tail 100

# Rollback if needed
docker service rollback maplefile_backend

Image Pull Authentication Error

# Re-authenticate
doctl registry login

# Retry
docker service update --image registry.digitalocean.com/ssp/maplefile-backend:prod maplefile_backend

Service Stuck Starting

# Common causes: database migrations failing, missing env vars, health check issues
# Check logs
docker service logs maplefile_backend --tail 50

# Rollback if urgent
docker service rollback maplefile_backend

Standard Deployment Workflow

# 1. Local: Build & push (note the git commit and Image ID)
cd ~/go/src/codeberg.org/mapleopentech/monorepo/cloud/maplefile-backend
task deploy
# Example output: "Deployed version d90b6e2b - use this to verify on production"

# Note the local Image ID for verification
docker images registry.digitalocean.com/ssp/maplefile-backend:prod
# Example: IMAGE ID = 74b2fafb1f69

# 2. Find which worker is running the service
ssh dockeradmin@<MANAGER_IP>
docker service ps maplefile_backend
# Note the worker node (e.g., mapleopentech-swarm-worker-8-prod)

# 3. Pull the new image on the MANAGER node
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod

# 4. Pull the new image on the WORKER node
ssh dockeradmin@<WORKER_NODE>
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod
exit

# 5. Force restart on manager
ssh dockeradmin@<MANAGER_IP>
docker service update --force maplefile_backend

# 6. Verify git commit matches what you deployed
docker service logs maplefile_backend --tail 50
# Look for: 📝 Git Commit: d90b6e2b...

Key points:

• You MUST pull the image on BOTH manager and worker nodes
• Use docker images to verify Image ID matches your local build
• Use docker service update --force to restart with the new image
• Check startup logs for Git Commit to verify correct version is running

---

Last Updated: November 2025