monorepo/cloud/infrastructure/production/operations/BACKEND_ACCESS.md

3.7 KiB

Backend Access & Database Operations

Access Backend Container

# Find which node runs the backend
ssh dockeradmin@<manager-ip>
docker service ps maplefile_backend --filter "desired-state=running"
# Note the NODE column

# SSH to that worker
ssh dockeradmin@<worker-ip>

# Get container ID
export BACKEND_CONTAINER=$(docker ps --filter "name=maplefile.*backend" -q | head -1)

# Open shell
docker exec -it $BACKEND_CONTAINER sh

# Or run single command
docker exec $BACKEND_CONTAINER ./maplefile-backend --help

View Logs

# Follow logs
docker logs -f $BACKEND_CONTAINER

# Last 100 lines
docker logs --tail 100 $BACKEND_CONTAINER

# Search for errors
docker logs $BACKEND_CONTAINER 2>&1 | grep -i error

Database Operations

Run Migrations (Safe)

docker exec $BACKEND_CONTAINER ./maplefile-backend migrate up

Auto-runs on backend startup when DATABASE_AUTO_MIGRATE=true (default in stack file).

Rollback Last Migration (Destructive)

docker exec $BACKEND_CONTAINER ./maplefile-backend migrate down

Only rolls back 1 migration. Run multiple times for multiple rollbacks.

Reset Database (Full Wipe)

# 1. SSH to any Cassandra node (any of the 3 nodes works)
ssh dockeradmin@<cassandra-node-ip>

# 2. Find the Cassandra container ID
export CASSANDRA_CONTAINER=$(docker ps --filter "name=cassandra" -q | head -1)

# 3. Drop keyspace (DELETES ALL DATA - propagates to all 3 nodes)
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "DROP KEYSPACE IF EXISTS maplefile;"

# 4. Wait for schema to propagate across cluster
sleep 5

# 5. Recreate keyspace (propagates to all 3 nodes)
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "
CREATE KEYSPACE IF NOT EXISTS maplefile
WITH replication = {
    'class': 'SimpleStrategy',
    'replication_factor': 3
};"

# 6. Wait for schema agreement across cluster
sleep 5

# 7. Verify keyspace exists
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "DESCRIBE KEYSPACE maplefile;"

# 8. Restart backend to run migrations
# You must pull the new image on the worker node first
# Find which worker runs the service:
ssh dockeradmin@<manager-ip>
docker service ps maplefile_backend
# Note the worker node name

# Pull image on the worker:
ssh dockeradmin@<worker-ip>
docker pull registry.digitalocean.com/ssp/maplefile-backend:prod
exit

# Force restart on manager:
ssh dockeradmin@<manager-ip>
docker service update --force maplefile_backend

# Verify new version is running:
docker service logs maplefile_backend --tail 50
# Look for: 📝 Git Commit: <commit-sha>

Troubleshooting

Container Not Found

# Check service status
docker service ps maplefile_backend

# List all backend containers
docker ps | grep backend

Wrong Container (MaplePress vs MapleFile)

# Verify you have MapleFile (not MaplePress)
docker ps | grep $BACKEND_CONTAINER
# Should show "maplefile-backend" in image name

Migration Fails

# Check environment (from worker node)
docker exec $BACKEND_CONTAINER env | grep DATABASE

# Check Cassandra connectivity
docker exec $BACKEND_CONTAINER nc -zv cassandra-1 9042

Configuration

Environment variables are in ~/stacks/maplefile-stack.yml on manager node, not .env files.

To change config:

  1. Edit ~/stacks/maplefile-stack.yml
  2. Pull new image on worker: ssh dockeradmin@<worker-ip> && docker pull registry.digitalocean.com/ssp/maplefile-backend:prod && exit
  3. Force restart on manager: docker service update --force maplefile_backend

Important: Worker nodes cache images locally. You MUST pull the new image on the worker node before restarting the service. The --resolve-image always and --with-registry-auth flags do NOT reliably force worker nodes to pull new images.


Last Updated: November 2025