monorepo/cloud/infrastructure/development/README.md

# 🏗️ MapleFile (Development) Infrastructure

> Shared development infrastructure for all MapleFile projects. Start once, use everywhere.

## 📖 What is this?

Think of this as your **local cloud environment**. Instead of each MapleFile project (maplefile-backend, maplepress-backend, etc.) running its own database, cache, and storage, they all share this common infrastructure - just like production apps share AWS/cloud services.

**What you get:**
- Database (Cassandra) - stores your data
- Cache (Redis) - makes things fast
- Search (Meilisearch) - powers search features
- File Storage (SeaweedFS) - stores uploaded files
- WordPress (for plugin testing)

**Why shared?**
- Start infrastructure once, restart your apps quickly (seconds vs minutes)
- Closer to real production setup
- Learn proper microservices architecture

**No environment variables needed here** - this project is already configured for local development. The apps that connect to it will have their own `.env` files.

## ⚡ TL;DR

```bash
task dev:start    # Start everything (takes 2-3 minutes first time)
task dev:status   # Verify all services show "healthy"
```

**Then:** Navigate to a backend project (`../maplepress-backend/` or `../maplefile-backend/`) and follow its README to set up and start the backend. See [What's Next?](#whats-next) section below.

## 📋 Prerequisites

You need these tools installed before starting. Don't worry - they're free and easy to install.

### 1. Docker Desktop

**What is Docker?** A tool that runs software in isolated containers. Think of it as lightweight virtual machines that start instantly.

**Download & Install:**
- **macOS:** [Docker Desktop for Mac](https://www.docker.com/products/docker-desktop/) (includes docker-compose)
- **Windows:** [Docker Desktop for Windows](https://www.docker.com/products/docker-desktop/)
- **Linux:** Follow instructions at [docs.docker.com/engine/install](https://docs.docker.com/engine/install/)

**Verify installation:**
```bash
docker --version          # Should show: Docker version 20.x or higher
docker compose version    # Should show: Docker Compose version 2.x or higher
```

**What is Docker Compose?** A tool for running multiple Docker containers together. It's **included with Docker Desktop** - you don't need to install it separately! When you install Docker Desktop, you automatically get Docker Compose.

**Note on Docker Compose versions:**
- **Docker Compose v1** (older): Uses `docker-compose` command (hyphen)
- **Docker Compose v2** (current): Uses `docker compose` command (space)
- Our Taskfile **automatically detects** which version you have and uses the correct command
- If you're on Linux with Docker Compose v2, use `docker compose version` (not `docker-compose --version`)

### 2. Task (Task Runner)

**What is Task?** A simple command runner (like `make` but better). We use it instead of typing long docker commands.

**Install:**
- **macOS:** `brew install go-task`
- **Windows:** `choco install go-task` (using [Chocolatey](https://chocolatey.org/))
- **Linux:** `snap install task --classic`
- **Manual install:** Download from [taskfile.dev](https://taskfile.dev/installation/)

**Verify installation:**
```bash
task --version  # Should show: Task version 3.x or higher
```

### 3. All other services (Cassandra, Redis, etc.)

**Do I need to install them?** **NO!** Docker will automatically download and run everything. You don't install Cassandra, Redis, or any database directly on your computer.

**What happens when you run `task dev:start`:**
1. Docker downloads required images (first time only - takes a few minutes)
2. Starts all services in containers
3. That's it - everything is ready to use!

## ❓ Common Questions

**Q: Do I need to configure environment variables or create a `.env` file?**
A: **No!** This infrastructure project is pre-configured for local development. However, the application projects that connect to it (like `maplefile-backend`) will need their own `.env` files - check their READMEs.

**Q: Do I need to install Cassandra, Redis, or other databases?**
A: **No!** Docker handles everything. You only install Docker and Task, nothing else.

**Q: Will this mess up my computer or conflict with other projects?**
A: **No!** Everything runs in isolated Docker containers. You can safely remove it all with `task dev:clean` and `docker system prune`.

**Q: How much disk space does this use?**
A: Initial download: ~2-3 GB. Running services + data: ~5-10 GB depending on usage.

**Q: Can I use this on Windows?**
A: **Yes!** Docker Desktop works on Windows. Just make sure to use PowerShell or Git Bash for commands.

**Q: What is Docker Compose? Do I need to install it separately?**
A: **No!** Docker Compose is included with Docker Desktop automatically. When you install Docker Desktop, you get both `docker` and `docker compose` commands.

**Q: I'm getting "docker-compose: command not found" on Linux. What should I do?**
A: You likely have Docker Compose v2, which uses `docker compose` (space) instead of `docker-compose` (hyphen). Our Taskfile automatically detects and uses the correct command. Just run `task dev:start` and it will work on both Mac and Linux.

## 🚀 Quick Start

### 1. Start Infrastructure

```bash
task dev:start
```

Wait for: `✅ Infrastructure ready!`

### 2. Verify Everything Works

```bash
task dev:status
```

**Expected output:** All services show `Up X minutes (healthy)`

```
NAMES                   STATUS                   PORTS
maple-cassandra-1-dev   Up 2 minutes (healthy)   0.0.0.0:9042->9042/tcp
maple-redis-dev         Up 2 minutes (healthy)   0.0.0.0:6379->6379/tcp
maple-wordpress-dev     Up 2 minutes (healthy)   0.0.0.0:8081->80/tcp
...
```

### 3. Start Your App

Now navigate to your app directory (e.g., `maplefile-backend`) and run its `task dev` command. Your app will automatically connect to this infrastructure.

### 4. Stop Infrastructure (End of Day)

```bash
task dev:stop    # Stops services, keeps data
```

## 🎯 What's Next?

🎉 **Infrastructure is running!** Now set up a backend:

- **MaplePress Backend:** [`../maplepress-backend/README.md`](../maplepress-backend/README.md)
- **MapleFile Backend:** [`../maplefile-backend/README.md`](../maplefile-backend/README.md)

Pick one, navigate to its directory, and follow its setup instructions.

## 📅 Daily Commands

```bash
# Morning - start infrastructure
task dev:start

# Check if everything is running
task dev:status

# Evening - stop infrastructure (keeps data)
task dev:stop

# Nuclear option - delete everything and start fresh
task dev:clean   # ⚠️ DELETES ALL DATA
```

## 🔍 Troubleshooting

### Service shows unhealthy or won't start

```bash
# Check logs for specific service
task dev:logs -- cassandra-1
task dev:logs -- redis
task dev:logs -- wordpress

# Or follow logs in real-time
task dev:logs -- cassandra-1
```

**Service names:** `cassandra-1`, `cassandra-2`, `cassandra-3`, `redis`, `meilisearch`, `seaweedfs`, `mariadb`, `wordpress`

### Port already in use

Another service is using the required ports. Check:
- Port 9042 (Cassandra)
- Port 6379 (Redis)
- Port 8081 (WordPress)
- Port 3306 (MariaDB)

Find and stop the conflicting service:
```bash
lsof -i :9042    # macOS/Linux
```

### Want to reset everything

```bash
task dev:clean   # Removes all containers and data
task dev:start   # Fresh start
```

## 🌐 What's Running?

When you start infrastructure, you get these services:

| Service | Port | Purpose | Access |
|---------|------|---------|--------|
| Cassandra Cluster | 9042 | Database (3-node cluster) | `task cql` |
| Redis | 6379 | Cache & sessions | `task redis` |
| Meilisearch | 7700 | Search engine | http://localhost:7700 |
| SeaweedFS | 8333, 9333 | S3-compatible storage | http://localhost:9333 |
| MariaDB | 3306 | WordPress database | - |
| WordPress | 8081 | Plugin testing | http://localhost:8081 |

## 🔧 Common Operations

### Working with Cassandra

```bash
# Open CQL shell
task cql

# List all keyspaces
task cql:keyspaces

# List tables in a keyspace
task cql:tables -- maplepress

# Check cluster health
task cql:status
```

**Available keyspaces:**
- `maplefile` - MapleFile backend (Redis DB: 1)
- `maplepress` - MaplePress backend (Redis DB: 0)

### Working with Redis

```bash
# Open Redis CLI
task redis

# Then inside Redis CLI:
# SELECT 0    # Switch to maplepress database
# SELECT 1    # Switch to maplefile database
# KEYS *      # List all keys
```

### Working with WordPress

**Access:** http://localhost:8081

**First-time setup:**
1. Visit http://localhost:8081
2. Complete WordPress installation wizard
3. Use any credentials (this is a dev site)

**Credentials for WordPress database:**
- Host: `mariadb:3306`
- Database: `wordpress`
- User: `wordpress`
- Password: `wordpress`

**View debug logs:**
```bash
docker exec -it maple-wordpress-dev tail -f /var/www/html/wp-content/debug.log
```

### Working with SeaweedFS (S3 Storage)

**Web UI:** http://localhost:9333

**S3 Configuration for your apps:**
```bash
S3_ENDPOINT=http://seaweedfs:8333
S3_REGION=us-east-1
S3_ACCESS_KEY=any
S3_SECRET_KEY=any
```

## 💻 Development Workflow

**Typical daily flow:**

1. **Morning:** `task dev:start` (in this directory)
2. **Start app:** `cd ../maplefile-backend && task dev`
3. **Work on code** - restart app as needed (fast!)
4. **Infrastructure keeps running** - no need to restart
5. **Evening:** `task dev:stop` (optional - can leave running)

**Why this approach?**
- Infrastructure takes 2-3 minutes to start (Cassandra cluster is slow)
- Your app restarts in seconds
- Start infrastructure once, restart apps freely

## 💾 Data Persistence

All data is stored in Docker volumes and survives restarts:

- `maple-cassandra-1-dev`, `maple-cassandra-2-dev`, `maple-cassandra-3-dev`
- `maple-redis-dev`
- `maple-meilisearch-dev`
- `maple-seaweedfs-dev`
- `maple-mariadb-dev`
- `maple-wordpress-dev`

**To completely reset (deletes all data):**
```bash
task dev:clean
```

## 🎓 Advanced Topics

> **⚠️ SKIP THIS SECTION FOR INITIAL SETUP!**
>
> These topics are for **future use** - after you've successfully set up and used the infrastructure. You don't need to read or do anything here when setting up for the first time.
>
> Come back here only when you need to:
> - Add a new project to the infrastructure (not needed now - mapleopentech and maplepress already configured)
> - Understand Cassandra cluster architecture (curiosity only)
> - Learn why we chose this approach (optional reading)

### Adding a New Project

**When do I need this?** Only if you're creating a brand new project (not maplefile-backend or maplepress-backend - those are already set up).

To add a new project to shared infrastructure:

1. Add keyspace to `cassandra/init-scripts/01-create-keyspaces.cql`:
```cql
CREATE KEYSPACE IF NOT EXISTS mynewproject
WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 3};
```

2. Configure your project's `docker-compose.dev.yml`:
```yaml
networks:
  maple-dev:
    external: true

services:
  app:
    environment:
      - DATABASE_HOSTS=cassandra-1:9042,cassandra-2:9042,cassandra-3:9042
      - DATABASE_KEYSPACE=mynewproject
      - DATABASE_CONSISTENCY=QUORUM
      - DATABASE_REPLICATION=3
      - REDIS_HOST=redis
      - REDIS_DB=2  # Use next available: 0=maplepress, 1=maplefile
    networks:
      - maple-dev
```

3. Restart infrastructure:
```bash
task dev:restart
```

### Cassandra Cluster Details

- **3-node cluster** for high availability
- **Replication factor: 3** (data on all nodes)
- **Consistency level: QUORUM** (2 of 3 nodes must agree)
- **Seed node:** cassandra-1 (other nodes join via this node)

### Architecture Decision: Why Separate Infrastructure?

**Benefits:**
- Faster app restarts (seconds vs minutes)
- Share infrastructure across multiple projects
- Closer to production architecture
- Learn proper service separation

**Trade-off:**
- One extra terminal/directory to manage
- Slightly more complex than monolithic docker-compose

We chose speed and realism over simplicity.

## Contributing

Found a bug? Want a feature to improve the infrastructure? Please create an [issue](https://codeberg.org/mapleopentech/monorepo/issues/new).

## License

This application is licensed under the [**GNU Affero General Public License v3.0**](https://opensource.org/license/agpl-v3). See [LICENSE](../../LICENSE) for more information.