153 lines
3.9 KiB
Markdown
153 lines
3.9 KiB
Markdown
# Database Migrations
|
|
|
|
This directory contains Cassandra CQL migrations for the MapleFile backend.
|
|
|
|
## ⚠️ Prerequisites: Keyspace Must Exist
|
|
|
|
**IMPORTANT:** Before running migrations, the `maplefile` keyspace must exist in Cassandra.
|
|
|
|
### Why Keyspaces Are Not in Migrations
|
|
|
|
Following industry best practices:
|
|
- **Keyspace creation = Infrastructure setup** (DevOps responsibility)
|
|
- **Table migrations = Application changes** (Backend responsibility)
|
|
|
|
This separation allows infrastructure decisions (replication strategy, topology) to be managed independently from application schema.
|
|
|
|
### Creating the Keyspace
|
|
|
|
**Development:**
|
|
```bash
|
|
cd cloud/infrastructure/development
|
|
|
|
# Find Cassandra container
|
|
export CASSANDRA_CONTAINER=$(docker ps --filter "name=cassandra" -q | head -1)
|
|
|
|
# Create keyspace
|
|
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "
|
|
CREATE KEYSPACE IF NOT EXISTS maplefile
|
|
WITH replication = {
|
|
'class': 'SimpleStrategy',
|
|
'replication_factor': 3
|
|
};"
|
|
```
|
|
|
|
**Production:**
|
|
See `cloud/infrastructure/production/setup/09_maplefile_backend.md` Step 9.3
|
|
|
|
## Auto-Migration
|
|
|
|
Migrations run **automatically on backend startup** when `DATABASE_AUTO_MIGRATE=true` (default).
|
|
|
|
The backend will:
|
|
1. **Expect the `maplefile` keyspace to exist** (created by DevOps)
|
|
2. Run all pending migrations in order (001, 002, 003, etc.)
|
|
3. Track migration state in Cassandra
|
|
4. Handle dirty migration states automatically
|
|
|
|
## Migration Files
|
|
|
|
Migrations use the `golang-migrate` tool format:
|
|
|
|
- **Up migrations**: `{version}_{description}.up.cql` - Applied when migrating forward
|
|
- **Down migrations**: `{version}_{description}.down.cql` - Applied when rolling back
|
|
|
|
### Current Migrations
|
|
|
|
- **001-024** - Table and index creation for sessions, users, files, collections, etc.
|
|
|
|
### Manual Migration
|
|
|
|
If you need to run migrations manually:
|
|
|
|
```bash
|
|
# Run all pending migrations
|
|
./maplefile-backend migrate up
|
|
|
|
# Rollback last migration
|
|
./maplefile-backend migrate down
|
|
|
|
# Check current version
|
|
./maplefile-backend migrate version
|
|
|
|
# Force version (fix dirty state)
|
|
./maplefile-backend migrate force <version>
|
|
```
|
|
|
|
### Disabling Auto-Migration
|
|
|
|
Set in `.env`:
|
|
```bash
|
|
DATABASE_AUTO_MIGRATE=false
|
|
```
|
|
|
|
## Creating New Migrations
|
|
|
|
1. Create new migration files with incremented version:
|
|
```bash
|
|
touch migrations/025_add_new_table.up.cql
|
|
touch migrations/025_add_new_table.down.cql
|
|
```
|
|
|
|
2. Write the CQL:
|
|
- **Up migration**: Create/modify schema
|
|
- **Down migration**: Reverse the changes
|
|
|
|
3. Commit both files
|
|
|
|
4. Restart backend or run `./maplefile-backend migrate up`
|
|
|
|
## Troubleshooting
|
|
|
|
### "keyspace does not exist" Error
|
|
|
|
**Cause**: The `maplefile` keyspace hasn't been created by DevOps/infrastructure setup
|
|
|
|
**Solution**:
|
|
```bash
|
|
# Development
|
|
cd cloud/infrastructure/development
|
|
export CASSANDRA_CONTAINER=$(docker ps --filter "name=cassandra" -q | head -1)
|
|
docker exec -it $CASSANDRA_CONTAINER cqlsh -e "
|
|
CREATE KEYSPACE IF NOT EXISTS maplefile
|
|
WITH replication = {
|
|
'class': 'SimpleStrategy',
|
|
'replication_factor': 3
|
|
};"
|
|
|
|
# Production
|
|
# See cloud/infrastructure/production/setup/09_maplefile_backend.md Step 9.3
|
|
|
|
# Then restart backend
|
|
docker service update --force maplefile_backend # Production
|
|
# Or: task dev:restart # Development
|
|
```
|
|
|
|
**Prevention**: Always create the keyspace before first backend deployment
|
|
|
|
### Dirty Migration State
|
|
|
|
**Symptom**: Backend won't start, logs show "dirty migration"
|
|
|
|
**Solution**:
|
|
```bash
|
|
# Force clean state at current version
|
|
./maplefile-backend migrate force <version>
|
|
|
|
# Then retry
|
|
./maplefile-backend migrate up
|
|
```
|
|
|
|
### Migration Failed
|
|
|
|
**Symptom**: Backend crashes during migration
|
|
|
|
**Solution**:
|
|
1. Check backend logs for specific CQL error
|
|
2. Fix the migration file
|
|
3. Force clean state: `./maplefile-backend migrate force <version-1>`
|
|
4. Restart backend or run `./maplefile-backend migrate up`
|
|
|
|
---
|
|
|
|
**Related**: See `pkg/storage/database/cassandradb/migration.go` for implementation
|