monorepo/cloud/maplefile-backend/migrations/README.md

Database Migrations

This directory contains Cassandra CQL migrations for the MapleFile backend.

⚠️ Prerequisites: Keyspace Must Exist

IMPORTANT: Before running migrations, the maplefile keyspace must exist in Cassandra.

Why Keyspaces Are Not in Migrations

Following industry best practices:

• Keyspace creation = Infrastructure setup (DevOps responsibility)
• Table migrations = Application changes (Backend responsibility)

This separation allows infrastructure decisions (replication strategy, topology) to be managed independently from application schema.

Creating the Keyspace

Development:

cd cloud/infrastructure/development

# Find Cassandra container
export CASSANDRA_CONTAINER=$(docker ps --filter "name=cassandra" -q | head -1)

# Create keyspace
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "
CREATE KEYSPACE IF NOT EXISTS maplefile
WITH replication = {
    'class': 'SimpleStrategy',
    'replication_factor': 3
};"

Production: See cloud/infrastructure/production/setup/09_maplefile_backend.md Step 9.3

Auto-Migration

Migrations run automatically on backend startup when DATABASE_AUTO_MIGRATE=true (default).

The backend will:

1. Expect the maplefile keyspace to exist (created by DevOps)
2. Run all pending migrations in order (001, 002, 003, etc.)
3. Track migration state in Cassandra
4. Handle dirty migration states automatically

Migration Files

Migrations use the golang-migrate tool format:

• Up migrations: {version}_{description}.up.cql - Applied when migrating forward
• Down migrations: {version}_{description}.down.cql - Applied when rolling back

Current Migrations

• 001-024 - Table and index creation for sessions, users, files, collections, etc.

Manual Migration

If you need to run migrations manually:

# Run all pending migrations
./maplefile-backend migrate up

# Rollback last migration
./maplefile-backend migrate down

# Check current version
./maplefile-backend migrate version

# Force version (fix dirty state)
./maplefile-backend migrate force <version>

Disabling Auto-Migration

Set in .env:

DATABASE_AUTO_MIGRATE=false

Creating New Migrations

1. Create new migration files with incremented version:

   touch migrations/025_add_new_table.up.cql
   touch migrations/025_add_new_table.down.cql
   

2. Write the CQL:

   • Up migration: Create/modify schema
   • Down migration: Reverse the changes

3. Commit both files

4. Restart backend or run ./maplefile-backend migrate up

Troubleshooting

"keyspace does not exist" Error

Cause: The maplefile keyspace hasn't been created by DevOps/infrastructure setup

Solution:

# Development
cd cloud/infrastructure/development
export CASSANDRA_CONTAINER=$(docker ps --filter "name=cassandra" -q | head -1)
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "
CREATE KEYSPACE IF NOT EXISTS maplefile
WITH replication = {
    'class': 'SimpleStrategy',
    'replication_factor': 3
};"

# Production
# See cloud/infrastructure/production/setup/09_maplefile_backend.md Step 9.3

# Then restart backend
docker service update --force maplefile_backend  # Production
# Or: task dev:restart  # Development

Prevention: Always create the keyspace before first backend deployment

Dirty Migration State

Symptom: Backend won't start, logs show "dirty migration"

Solution:

# Force clean state at current version
./maplefile-backend migrate force <version>

# Then retry
./maplefile-backend migrate up

Migration Failed

Symptom: Backend crashes during migration

Solution:

1. Check backend logs for specific CQL error
2. Fix the migration file
3. Force clean state: ./maplefile-backend migrate force <version-1>
4. Restart backend or run ./maplefile-backend migrate up

---

Related: See pkg/storage/database/cassandradb/migration.go for implementation