Initial commit: Open sourcing all of the Maple Open Technologies code.
This commit is contained in:
commit
755d54a99d
2010 changed files with 448675 additions and 0 deletions
387
cloud/infrastructure/development/README.md
Normal file
387
cloud/infrastructure/development/README.md
Normal file
|
|
@ -0,0 +1,387 @@
|
|||
# 🏗️ MapleFile (Development) Infrastructure
|
||||
|
||||
> Shared development infrastructure for all MapleFile projects. Start once, use everywhere.
|
||||
|
||||
## 📖 What is this?
|
||||
|
||||
Think of this as your **local cloud environment**. Instead of each MapleFile project (maplefile-backend, maplepress-backend, etc.) running its own database, cache, and storage, they all share this common infrastructure - just like production apps share AWS/cloud services.
|
||||
|
||||
**What you get:**
|
||||
- Database (Cassandra) - stores your data
|
||||
- Cache (Redis) - makes things fast
|
||||
- Search (Meilisearch) - powers search features
|
||||
- File Storage (SeaweedFS) - stores uploaded files
|
||||
- WordPress (for plugin testing)
|
||||
|
||||
**Why shared?**
|
||||
- Start infrastructure once, restart your apps quickly (seconds vs minutes)
|
||||
- Closer to real production setup
|
||||
- Learn proper microservices architecture
|
||||
|
||||
**No environment variables needed here** - this project is already configured for local development. The apps that connect to it will have their own `.env` files.
|
||||
|
||||
## ⚡ TL;DR
|
||||
|
||||
```bash
|
||||
task dev:start # Start everything (takes 2-3 minutes first time)
|
||||
task dev:status # Verify all services show "healthy"
|
||||
```
|
||||
|
||||
**Then:** Navigate to a backend project (`../maplepress-backend/` or `../maplefile-backend/`) and follow its README to set up and start the backend. See [What's Next?](#whats-next) section below.
|
||||
|
||||
## 📋 Prerequisites
|
||||
|
||||
You need these tools installed before starting. Don't worry - they're free and easy to install.
|
||||
|
||||
### 1. Docker Desktop
|
||||
|
||||
**What is Docker?** A tool that runs software in isolated containers. Think of it as lightweight virtual machines that start instantly.
|
||||
|
||||
**Download & Install:**
|
||||
- **macOS:** [Docker Desktop for Mac](https://www.docker.com/products/docker-desktop/) (includes docker-compose)
|
||||
- **Windows:** [Docker Desktop for Windows](https://www.docker.com/products/docker-desktop/)
|
||||
- **Linux:** Follow instructions at [docs.docker.com/engine/install](https://docs.docker.com/engine/install/)
|
||||
|
||||
**Verify installation:**
|
||||
```bash
|
||||
docker --version # Should show: Docker version 20.x or higher
|
||||
docker compose version # Should show: Docker Compose version 2.x or higher
|
||||
```
|
||||
|
||||
**What is Docker Compose?** A tool for running multiple Docker containers together. It's **included with Docker Desktop** - you don't need to install it separately! When you install Docker Desktop, you automatically get Docker Compose.
|
||||
|
||||
**Note on Docker Compose versions:**
|
||||
- **Docker Compose v1** (older): Uses `docker-compose` command (hyphen)
|
||||
- **Docker Compose v2** (current): Uses `docker compose` command (space)
|
||||
- Our Taskfile **automatically detects** which version you have and uses the correct command
|
||||
- If you're on Linux with Docker Compose v2, use `docker compose version` (not `docker-compose --version`)
|
||||
|
||||
### 2. Task (Task Runner)
|
||||
|
||||
**What is Task?** A simple command runner (like `make` but better). We use it instead of typing long docker commands.
|
||||
|
||||
**Install:**
|
||||
- **macOS:** `brew install go-task`
|
||||
- **Windows:** `choco install go-task` (using [Chocolatey](https://chocolatey.org/))
|
||||
- **Linux:** `snap install task --classic`
|
||||
- **Manual install:** Download from [taskfile.dev](https://taskfile.dev/installation/)
|
||||
|
||||
**Verify installation:**
|
||||
```bash
|
||||
task --version # Should show: Task version 3.x or higher
|
||||
```
|
||||
|
||||
### 3. All other services (Cassandra, Redis, etc.)
|
||||
|
||||
**Do I need to install them?** **NO!** Docker will automatically download and run everything. You don't install Cassandra, Redis, or any database directly on your computer.
|
||||
|
||||
**What happens when you run `task dev:start`:**
|
||||
1. Docker downloads required images (first time only - takes a few minutes)
|
||||
2. Starts all services in containers
|
||||
3. That's it - everything is ready to use!
|
||||
|
||||
## ❓ Common Questions
|
||||
|
||||
**Q: Do I need to configure environment variables or create a `.env` file?**
|
||||
A: **No!** This infrastructure project is pre-configured for local development. However, the application projects that connect to it (like `maplefile-backend`) will need their own `.env` files - check their READMEs.
|
||||
|
||||
**Q: Do I need to install Cassandra, Redis, or other databases?**
|
||||
A: **No!** Docker handles everything. You only install Docker and Task, nothing else.
|
||||
|
||||
**Q: Will this mess up my computer or conflict with other projects?**
|
||||
A: **No!** Everything runs in isolated Docker containers. You can safely remove it all with `task dev:clean` and `docker system prune`.
|
||||
|
||||
**Q: How much disk space does this use?**
|
||||
A: Initial download: ~2-3 GB. Running services + data: ~5-10 GB depending on usage.
|
||||
|
||||
**Q: Can I use this on Windows?**
|
||||
A: **Yes!** Docker Desktop works on Windows. Just make sure to use PowerShell or Git Bash for commands.
|
||||
|
||||
**Q: What is Docker Compose? Do I need to install it separately?**
|
||||
A: **No!** Docker Compose is included with Docker Desktop automatically. When you install Docker Desktop, you get both `docker` and `docker compose` commands.
|
||||
|
||||
**Q: I'm getting "docker-compose: command not found" on Linux. What should I do?**
|
||||
A: You likely have Docker Compose v2, which uses `docker compose` (space) instead of `docker-compose` (hyphen). Our Taskfile automatically detects and uses the correct command. Just run `task dev:start` and it will work on both Mac and Linux.
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### 1. Start Infrastructure
|
||||
|
||||
```bash
|
||||
task dev:start
|
||||
```
|
||||
|
||||
Wait for: `✅ Infrastructure ready!`
|
||||
|
||||
### 2. Verify Everything Works
|
||||
|
||||
```bash
|
||||
task dev:status
|
||||
```
|
||||
|
||||
**Expected output:** All services show `Up X minutes (healthy)`
|
||||
|
||||
```
|
||||
NAMES STATUS PORTS
|
||||
maple-cassandra-1-dev Up 2 minutes (healthy) 0.0.0.0:9042->9042/tcp
|
||||
maple-redis-dev Up 2 minutes (healthy) 0.0.0.0:6379->6379/tcp
|
||||
maple-wordpress-dev Up 2 minutes (healthy) 0.0.0.0:8081->80/tcp
|
||||
...
|
||||
```
|
||||
|
||||
### 3. Start Your App
|
||||
|
||||
Now navigate to your app directory (e.g., `maplefile-backend`) and run its `task dev` command. Your app will automatically connect to this infrastructure.
|
||||
|
||||
### 4. Stop Infrastructure (End of Day)
|
||||
|
||||
```bash
|
||||
task dev:stop # Stops services, keeps data
|
||||
```
|
||||
|
||||
## 🎯 What's Next?
|
||||
|
||||
🎉 **Infrastructure is running!** Now set up a backend:
|
||||
|
||||
- **MaplePress Backend:** [`../maplepress-backend/README.md`](../maplepress-backend/README.md)
|
||||
- **MapleFile Backend:** [`../maplefile-backend/README.md`](../maplefile-backend/README.md)
|
||||
|
||||
Pick one, navigate to its directory, and follow its setup instructions.
|
||||
|
||||
## 📅 Daily Commands
|
||||
|
||||
```bash
|
||||
# Morning - start infrastructure
|
||||
task dev:start
|
||||
|
||||
# Check if everything is running
|
||||
task dev:status
|
||||
|
||||
# Evening - stop infrastructure (keeps data)
|
||||
task dev:stop
|
||||
|
||||
# Nuclear option - delete everything and start fresh
|
||||
task dev:clean # ⚠️ DELETES ALL DATA
|
||||
```
|
||||
|
||||
## 🔍 Troubleshooting
|
||||
|
||||
### Service shows unhealthy or won't start
|
||||
|
||||
```bash
|
||||
# Check logs for specific service
|
||||
task dev:logs -- cassandra-1
|
||||
task dev:logs -- redis
|
||||
task dev:logs -- wordpress
|
||||
|
||||
# Or follow logs in real-time
|
||||
task dev:logs -- cassandra-1
|
||||
```
|
||||
|
||||
**Service names:** `cassandra-1`, `cassandra-2`, `cassandra-3`, `redis`, `meilisearch`, `seaweedfs`, `mariadb`, `wordpress`
|
||||
|
||||
### Port already in use
|
||||
|
||||
Another service is using the required ports. Check:
|
||||
- Port 9042 (Cassandra)
|
||||
- Port 6379 (Redis)
|
||||
- Port 8081 (WordPress)
|
||||
- Port 3306 (MariaDB)
|
||||
|
||||
Find and stop the conflicting service:
|
||||
```bash
|
||||
lsof -i :9042 # macOS/Linux
|
||||
```
|
||||
|
||||
### Want to reset everything
|
||||
|
||||
```bash
|
||||
task dev:clean # Removes all containers and data
|
||||
task dev:start # Fresh start
|
||||
```
|
||||
|
||||
## 🌐 What's Running?
|
||||
|
||||
When you start infrastructure, you get these services:
|
||||
|
||||
| Service | Port | Purpose | Access |
|
||||
|---------|------|---------|--------|
|
||||
| Cassandra Cluster | 9042 | Database (3-node cluster) | `task cql` |
|
||||
| Redis | 6379 | Cache & sessions | `task redis` |
|
||||
| Meilisearch | 7700 | Search engine | http://localhost:7700 |
|
||||
| SeaweedFS | 8333, 9333 | S3-compatible storage | http://localhost:9333 |
|
||||
| MariaDB | 3306 | WordPress database | - |
|
||||
| WordPress | 8081 | Plugin testing | http://localhost:8081 |
|
||||
|
||||
## 🔧 Common Operations
|
||||
|
||||
### Working with Cassandra
|
||||
|
||||
```bash
|
||||
# Open CQL shell
|
||||
task cql
|
||||
|
||||
# List all keyspaces
|
||||
task cql:keyspaces
|
||||
|
||||
# List tables in a keyspace
|
||||
task cql:tables -- maplepress
|
||||
|
||||
# Check cluster health
|
||||
task cql:status
|
||||
```
|
||||
|
||||
**Available keyspaces:**
|
||||
- `maplefile` - MapleFile backend (Redis DB: 1)
|
||||
- `maplepress` - MaplePress backend (Redis DB: 0)
|
||||
|
||||
### Working with Redis
|
||||
|
||||
```bash
|
||||
# Open Redis CLI
|
||||
task redis
|
||||
|
||||
# Then inside Redis CLI:
|
||||
# SELECT 0 # Switch to maplepress database
|
||||
# SELECT 1 # Switch to maplefile database
|
||||
# KEYS * # List all keys
|
||||
```
|
||||
|
||||
### Working with WordPress
|
||||
|
||||
**Access:** http://localhost:8081
|
||||
|
||||
**First-time setup:**
|
||||
1. Visit http://localhost:8081
|
||||
2. Complete WordPress installation wizard
|
||||
3. Use any credentials (this is a dev site)
|
||||
|
||||
**Credentials for WordPress database:**
|
||||
- Host: `mariadb:3306`
|
||||
- Database: `wordpress`
|
||||
- User: `wordpress`
|
||||
- Password: `wordpress`
|
||||
|
||||
**View debug logs:**
|
||||
```bash
|
||||
docker exec -it maple-wordpress-dev tail -f /var/www/html/wp-content/debug.log
|
||||
```
|
||||
|
||||
### Working with SeaweedFS (S3 Storage)
|
||||
|
||||
**Web UI:** http://localhost:9333
|
||||
|
||||
**S3 Configuration for your apps:**
|
||||
```bash
|
||||
S3_ENDPOINT=http://seaweedfs:8333
|
||||
S3_REGION=us-east-1
|
||||
S3_ACCESS_KEY=any
|
||||
S3_SECRET_KEY=any
|
||||
```
|
||||
|
||||
## 💻 Development Workflow
|
||||
|
||||
**Typical daily flow:**
|
||||
|
||||
1. **Morning:** `task dev:start` (in this directory)
|
||||
2. **Start app:** `cd ../maplefile-backend && task dev`
|
||||
3. **Work on code** - restart app as needed (fast!)
|
||||
4. **Infrastructure keeps running** - no need to restart
|
||||
5. **Evening:** `task dev:stop` (optional - can leave running)
|
||||
|
||||
**Why this approach?**
|
||||
- Infrastructure takes 2-3 minutes to start (Cassandra cluster is slow)
|
||||
- Your app restarts in seconds
|
||||
- Start infrastructure once, restart apps freely
|
||||
|
||||
## 💾 Data Persistence
|
||||
|
||||
All data is stored in Docker volumes and survives restarts:
|
||||
|
||||
- `maple-cassandra-1-dev`, `maple-cassandra-2-dev`, `maple-cassandra-3-dev`
|
||||
- `maple-redis-dev`
|
||||
- `maple-meilisearch-dev`
|
||||
- `maple-seaweedfs-dev`
|
||||
- `maple-mariadb-dev`
|
||||
- `maple-wordpress-dev`
|
||||
|
||||
**To completely reset (deletes all data):**
|
||||
```bash
|
||||
task dev:clean
|
||||
```
|
||||
|
||||
## 🎓 Advanced Topics
|
||||
|
||||
> **⚠️ SKIP THIS SECTION FOR INITIAL SETUP!**
|
||||
>
|
||||
> These topics are for **future use** - after you've successfully set up and used the infrastructure. You don't need to read or do anything here when setting up for the first time.
|
||||
>
|
||||
> Come back here only when you need to:
|
||||
> - Add a new project to the infrastructure (not needed now - mapleopentech and maplepress already configured)
|
||||
> - Understand Cassandra cluster architecture (curiosity only)
|
||||
> - Learn why we chose this approach (optional reading)
|
||||
|
||||
### Adding a New Project
|
||||
|
||||
**When do I need this?** Only if you're creating a brand new project (not maplefile-backend or maplepress-backend - those are already set up).
|
||||
|
||||
To add a new project to shared infrastructure:
|
||||
|
||||
1. Add keyspace to `cassandra/init-scripts/01-create-keyspaces.cql`:
|
||||
```cql
|
||||
CREATE KEYSPACE IF NOT EXISTS mynewproject
|
||||
WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 3};
|
||||
```
|
||||
|
||||
2. Configure your project's `docker-compose.dev.yml`:
|
||||
```yaml
|
||||
networks:
|
||||
maple-dev:
|
||||
external: true
|
||||
|
||||
services:
|
||||
app:
|
||||
environment:
|
||||
- DATABASE_HOSTS=cassandra-1:9042,cassandra-2:9042,cassandra-3:9042
|
||||
- DATABASE_KEYSPACE=mynewproject
|
||||
- DATABASE_CONSISTENCY=QUORUM
|
||||
- DATABASE_REPLICATION=3
|
||||
- REDIS_HOST=redis
|
||||
- REDIS_DB=2 # Use next available: 0=maplepress, 1=maplefile
|
||||
networks:
|
||||
- maple-dev
|
||||
```
|
||||
|
||||
3. Restart infrastructure:
|
||||
```bash
|
||||
task dev:restart
|
||||
```
|
||||
|
||||
### Cassandra Cluster Details
|
||||
|
||||
- **3-node cluster** for high availability
|
||||
- **Replication factor: 3** (data on all nodes)
|
||||
- **Consistency level: QUORUM** (2 of 3 nodes must agree)
|
||||
- **Seed node:** cassandra-1 (other nodes join via this node)
|
||||
|
||||
### Architecture Decision: Why Separate Infrastructure?
|
||||
|
||||
**Benefits:**
|
||||
- Faster app restarts (seconds vs minutes)
|
||||
- Share infrastructure across multiple projects
|
||||
- Closer to production architecture
|
||||
- Learn proper service separation
|
||||
|
||||
**Trade-off:**
|
||||
- One extra terminal/directory to manage
|
||||
- Slightly more complex than monolithic docker-compose
|
||||
|
||||
We chose speed and realism over simplicity.
|
||||
|
||||
## Contributing
|
||||
|
||||
Found a bug? Want a feature to improve the infrastructure? Please create an [issue](https://codeberg.org/mapleopentech/monorepo/issues/new).
|
||||
|
||||
## License
|
||||
|
||||
This application is licensed under the [**GNU Affero General Public License v3.0**](https://opensource.org/license/agpl-v3). See [LICENSE](../../LICENSE) for more information.
|
||||
168
cloud/infrastructure/development/Taskfile.yml
Normal file
168
cloud/infrastructure/development/Taskfile.yml
Normal file
|
|
@ -0,0 +1,168 @@
|
|||
version: '3'
|
||||
|
||||
# Variables for Docker Compose command detection
|
||||
vars:
|
||||
DOCKER_COMPOSE_CMD:
|
||||
sh: |
|
||||
if command -v docker-compose >/dev/null 2>&1; then
|
||||
echo "docker-compose"
|
||||
elif docker compose version >/dev/null 2>&1; then
|
||||
echo "docker compose"
|
||||
else
|
||||
echo "docker-compose"
|
||||
fi
|
||||
|
||||
tasks:
|
||||
dev:start:
|
||||
desc: Start all infrastructure services for development
|
||||
cmds:
|
||||
- "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml up -d"
|
||||
- echo "⏳ Waiting for services to be healthy..."
|
||||
- task: dev:wait
|
||||
- task: dev:init
|
||||
- echo ""
|
||||
- echo "✅ Infrastructure ready!"
|
||||
- echo ""
|
||||
- echo "📊 Running Services:"
|
||||
- docker ps --filter "name=maple-"
|
||||
|
||||
dev:wait:
|
||||
desc: Wait for all services to be healthy
|
||||
silent: true
|
||||
cmds:
|
||||
- |
|
||||
echo "Waiting for Cassandra Node 1..."
|
||||
for i in {1..30}; do
|
||||
if docker exec maple-cassandra-1-dev cqlsh -e "describe cluster" >/dev/null 2>&1; then
|
||||
echo "✅ Cassandra Node 1 is ready"
|
||||
break
|
||||
fi
|
||||
echo " ... ($i/30)"
|
||||
sleep 2
|
||||
done
|
||||
- |
|
||||
echo "Waiting for Cassandra Node 2..."
|
||||
for i in {1..30}; do
|
||||
if docker exec maple-cassandra-2-dev cqlsh -e "describe cluster" >/dev/null 2>&1; then
|
||||
echo "✅ Cassandra Node 2 is ready"
|
||||
break
|
||||
fi
|
||||
echo " ... ($i/30)"
|
||||
sleep 2
|
||||
done
|
||||
- |
|
||||
echo "Waiting for Cassandra Node 3..."
|
||||
for i in {1..30}; do
|
||||
if docker exec maple-cassandra-3-dev cqlsh -e "describe cluster" >/dev/null 2>&1; then
|
||||
echo "✅ Cassandra Node 3 is ready"
|
||||
break
|
||||
fi
|
||||
echo " ... ($i/30)"
|
||||
sleep 2
|
||||
done
|
||||
- |
|
||||
echo "Waiting for Redis..."
|
||||
for i in {1..10}; do
|
||||
if docker exec maple-redis-dev redis-cli ping >/dev/null 2>&1; then
|
||||
echo "✅ Redis is ready"
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
- |
|
||||
echo "Waiting for SeaweedFS..."
|
||||
for i in {1..10}; do
|
||||
if docker exec maple-seaweedfs-dev /usr/bin/wget -q --spider http://127.0.0.1:9333/cluster/status 2>/dev/null; then
|
||||
echo "✅ SeaweedFS is ready"
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
dev:init:
|
||||
desc: Initialize keyspaces and databases
|
||||
cmds:
|
||||
- |
|
||||
echo "📦 Initializing Cassandra keyspaces..."
|
||||
docker exec -i maple-cassandra-1-dev cqlsh < cassandra/init-scripts/01-create-keyspaces.cql
|
||||
echo "✅ Keyspaces initialized with replication_factor=3"
|
||||
|
||||
dev:status:
|
||||
desc: Show status of all infrastructure services
|
||||
cmds:
|
||||
- |
|
||||
echo "📊 Infrastructure Status:"
|
||||
docker ps --filter "name=maple-" --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"
|
||||
|
||||
dev:stop:
|
||||
desc: Stop all infrastructure services (keeps data)
|
||||
cmds:
|
||||
- "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml down"
|
||||
- echo "✅ Infrastructure stopped (data preserved)"
|
||||
|
||||
dev:restart:
|
||||
desc: Restart all infrastructure services
|
||||
cmds:
|
||||
- task: dev:stop
|
||||
- task: dev:start
|
||||
|
||||
dev:logs:
|
||||
desc: View infrastructure logs (usage task dev:logs -- cassandra)
|
||||
cmds:
|
||||
- "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml logs -f {{.CLI_ARGS}}"
|
||||
|
||||
dev:clean:
|
||||
desc: Stop services and remove all data (DESTRUCTIVE!)
|
||||
prompt: This will DELETE ALL DATA in Cassandra, Redis, Meilisearch, and SeaweedFS. Continue?
|
||||
cmds:
|
||||
- "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml down -v"
|
||||
- echo "✅ Infrastructure cleaned (all data removed)"
|
||||
|
||||
dev:clean:keyspace:
|
||||
desc: Drop and recreate a specific Cassandra keyspace (usage task dev:clean:keyspace -- maplefile)
|
||||
prompt: This will DELETE ALL DATA in the {{.CLI_ARGS}} keyspace. Continue?
|
||||
cmds:
|
||||
- |
|
||||
KEYSPACE={{.CLI_ARGS}}
|
||||
if [ -z "$KEYSPACE" ]; then
|
||||
echo "❌ Error: Please specify a keyspace name"
|
||||
echo "Usage: task dev:clean:keyspace -- maplefile"
|
||||
exit 1
|
||||
fi
|
||||
echo "🗑️ Dropping keyspace: $KEYSPACE"
|
||||
docker exec maple-cassandra-1-dev cqlsh -e "DROP KEYSPACE IF EXISTS $KEYSPACE;"
|
||||
echo "📦 Recreating keyspace: $KEYSPACE"
|
||||
docker exec maple-cassandra-1-dev cqlsh -e "CREATE KEYSPACE IF NOT EXISTS $KEYSPACE WITH REPLICATION = {'class': 'SimpleStrategy', 'replication_factor': 3} AND DURABLE_WRITES = true;"
|
||||
echo "✅ Keyspace $KEYSPACE cleaned and recreated"
|
||||
|
||||
# Cassandra-specific tasks
|
||||
cql:
|
||||
desc: Open Cassandra CQL shell (connects to node 1)
|
||||
cmds:
|
||||
- docker exec -it maple-cassandra-1-dev cqlsh
|
||||
|
||||
cql:keyspaces:
|
||||
desc: List all keyspaces
|
||||
cmds:
|
||||
- docker exec maple-cassandra-1-dev cqlsh -e "DESCRIBE KEYSPACES;"
|
||||
|
||||
cql:tables:
|
||||
desc: List tables in a keyspace (usage task cql:tables -- maplepress)
|
||||
cmds:
|
||||
- docker exec maple-cassandra-1-dev cqlsh -e "USE {{.CLI_ARGS}}; DESCRIBE TABLES;"
|
||||
|
||||
cql:status:
|
||||
desc: Show Cassandra cluster status
|
||||
cmds:
|
||||
- docker exec maple-cassandra-1-dev nodetool status
|
||||
|
||||
# Redis-specific tasks
|
||||
redis:
|
||||
desc: Open Redis CLI
|
||||
cmds:
|
||||
- docker exec -it maple-redis-dev redis-cli
|
||||
|
||||
redis:info:
|
||||
desc: Show Redis info
|
||||
cmds:
|
||||
- docker exec maple-redis-dev redis-cli INFO
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
-- Maple Infrastructure - Keyspace Initialization
|
||||
-- This creates keyspaces for all Maple projects with replication factor 3
|
||||
|
||||
-- MaplePress Backend
|
||||
CREATE KEYSPACE IF NOT EXISTS maplepress
|
||||
WITH REPLICATION = {
|
||||
'class': 'SimpleStrategy',
|
||||
'replication_factor': 3
|
||||
}
|
||||
AND DURABLE_WRITES = true;
|
||||
|
||||
|
||||
-- MapleFile Backend
|
||||
CREATE KEYSPACE IF NOT EXISTS maplefile
|
||||
WITH REPLICATION = {
|
||||
'class': 'SimpleStrategy',
|
||||
'replication_factor': 3
|
||||
}
|
||||
AND DURABLE_WRITES = true;
|
||||
|
||||
-- Future projects can be added here
|
||||
-- Example:
|
||||
-- CREATE KEYSPACE IF NOT EXISTS mapleanalytics
|
||||
-- WITH REPLICATION = {
|
||||
-- 'class': 'SimpleStrategy',
|
||||
-- 'replication_factor': 1
|
||||
-- };
|
||||
|
||||
-- Verify keyspaces were created
|
||||
DESCRIBE KEYSPACES;
|
||||
250
cloud/infrastructure/development/docker-compose.dev.yml
Normal file
250
cloud/infrastructure/development/docker-compose.dev.yml
Normal file
|
|
@ -0,0 +1,250 @@
|
|||
# Shared network for all Maple services in development
|
||||
networks:
|
||||
maple-dev:
|
||||
name: maple-dev
|
||||
driver: bridge
|
||||
|
||||
# Persistent volumes for development data
|
||||
volumes:
|
||||
cassandra-1-dev-data:
|
||||
name: maple-cassandra-1-dev
|
||||
cassandra-2-dev-data:
|
||||
name: maple-cassandra-2-dev
|
||||
cassandra-3-dev-data:
|
||||
name: maple-cassandra-3-dev
|
||||
redis-dev-data:
|
||||
name: maple-redis-dev
|
||||
meilisearch-dev-data:
|
||||
name: maple-meilisearch-dev
|
||||
seaweedfs-dev-data:
|
||||
name: maple-seaweedfs-dev
|
||||
mariadb-dev-data:
|
||||
name: maple-mariadb-dev
|
||||
wordpress-dev-data:
|
||||
name: maple-wordpress-dev
|
||||
|
||||
services:
|
||||
cassandra-1:
|
||||
image: cassandra:5.0.4
|
||||
container_name: maple-cassandra-1-dev
|
||||
hostname: cassandra-1
|
||||
ports:
|
||||
- "9042:9042" # CQL native transport
|
||||
- "9160:9160" # Thrift (legacy, optional)
|
||||
environment:
|
||||
- CASSANDRA_CLUSTER_NAME=maple-dev-cluster
|
||||
- CASSANDRA_DC=datacenter1
|
||||
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
|
||||
- CASSANDRA_SEEDS=cassandra-1,cassandra-2,cassandra-3
|
||||
- MAX_HEAP_SIZE=512M
|
||||
- HEAP_NEWSIZE=128M
|
||||
volumes:
|
||||
- cassandra-1-dev-data:/var/lib/cassandra
|
||||
- ./cassandra/init-scripts:/init-scripts:ro
|
||||
networks:
|
||||
- maple-dev
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "cqlsh -e 'describe cluster' || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 80s
|
||||
restart: unless-stopped
|
||||
|
||||
cassandra-2:
|
||||
image: cassandra:5.0.4
|
||||
container_name: maple-cassandra-2-dev
|
||||
hostname: cassandra-2
|
||||
environment:
|
||||
- CASSANDRA_CLUSTER_NAME=maple-dev-cluster
|
||||
- CASSANDRA_DC=datacenter1
|
||||
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
|
||||
- CASSANDRA_SEEDS=cassandra-1,cassandra-2,cassandra-3
|
||||
- MAX_HEAP_SIZE=512M
|
||||
- HEAP_NEWSIZE=128M
|
||||
volumes:
|
||||
- cassandra-2-dev-data:/var/lib/cassandra
|
||||
networks:
|
||||
- maple-dev
|
||||
depends_on:
|
||||
- cassandra-1
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "cqlsh -e 'describe cluster' || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 80s
|
||||
restart: unless-stopped
|
||||
|
||||
cassandra-3:
|
||||
image: cassandra:5.0.4
|
||||
container_name: maple-cassandra-3-dev
|
||||
hostname: cassandra-3
|
||||
environment:
|
||||
- CASSANDRA_CLUSTER_NAME=maple-dev-cluster
|
||||
- CASSANDRA_DC=datacenter1
|
||||
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
|
||||
- CASSANDRA_SEEDS=cassandra-1,cassandra-2,cassandra-3
|
||||
- MAX_HEAP_SIZE=512M
|
||||
- HEAP_NEWSIZE=128M
|
||||
volumes:
|
||||
- cassandra-3-dev-data:/var/lib/cassandra
|
||||
networks:
|
||||
- maple-dev
|
||||
depends_on:
|
||||
- cassandra-1
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "cqlsh -e 'describe cluster' || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 5
|
||||
start_period: 80s
|
||||
restart: unless-stopped
|
||||
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
container_name: maple-redis-dev
|
||||
hostname: redis
|
||||
ports:
|
||||
- "6379:6379"
|
||||
volumes:
|
||||
- redis-dev-data:/data
|
||||
- ./redis/redis.dev.conf:/usr/local/etc/redis/redis.conf:ro
|
||||
networks:
|
||||
- maple-dev
|
||||
command: redis-server /usr/local/etc/redis/redis.conf
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
|
||||
meilisearch:
|
||||
image: getmeili/meilisearch:v1.5
|
||||
container_name: maple-meilisearch-dev
|
||||
hostname: meilisearch
|
||||
ports:
|
||||
- "7700:7700"
|
||||
environment:
|
||||
- MEILI_ENV=development
|
||||
- MEILI_MASTER_KEY=maple-dev-master-key-change-in-production
|
||||
- MEILI_NO_ANALYTICS=true
|
||||
volumes:
|
||||
- meilisearch-dev-data:/meili_data
|
||||
networks:
|
||||
- maple-dev
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:7700/health"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
|
||||
seaweedfs:
|
||||
image: chrislusf/seaweedfs:latest
|
||||
container_name: maple-seaweedfs-dev
|
||||
hostname: seaweedfs
|
||||
ports:
|
||||
- "8333:8333" # S3 API
|
||||
- "9333:9333" # Master server (web UI)
|
||||
- "8080:8080" # Volume server
|
||||
environment:
|
||||
- WEED_MASTER_VOLUME_SIZE_LIMIT_MB=1024
|
||||
volumes:
|
||||
- seaweedfs-dev-data:/data
|
||||
networks:
|
||||
- maple-dev
|
||||
command: server -s3 -dir=/data -s3.port=8333 -ip=0.0.0.0
|
||||
healthcheck:
|
||||
test: ["CMD", "/usr/bin/wget", "-q", "--spider", "http://127.0.0.1:9333/cluster/status"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
start_period: 15s
|
||||
restart: unless-stopped
|
||||
|
||||
# Nginx - CORS proxy for SeaweedFS
|
||||
# Access: localhost:8334 (proxies to seaweedfs:8333 with CORS headers)
|
||||
# Use this endpoint from frontend for file uploads
|
||||
nginx-s3-proxy:
|
||||
image: nginx:alpine
|
||||
container_name: maple-nginx-s3-proxy-dev
|
||||
hostname: nginx-s3-proxy
|
||||
ports:
|
||||
- "8334:8334" # CORS-enabled S3 API proxy
|
||||
volumes:
|
||||
- ./nginx/seaweedfs-cors.conf:/etc/nginx/conf.d/default.conf:ro
|
||||
networks:
|
||||
- maple-dev
|
||||
depends_on:
|
||||
- seaweedfs
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "--spider", "http://localhost:8334/"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 3
|
||||
restart: unless-stopped
|
||||
|
||||
# MariaDB - WordPress database
|
||||
# Access: localhost:3306
|
||||
# Credentials: wordpress/wordpress (root: maple-dev-root-password)
|
||||
mariadb:
|
||||
image: mariadb:11.2
|
||||
container_name: maple-mariadb-dev
|
||||
hostname: mariadb
|
||||
ports:
|
||||
- "3306:3306"
|
||||
environment:
|
||||
- MARIADB_ROOT_PASSWORD=maple-dev-root-password
|
||||
- MARIADB_DATABASE=wordpress
|
||||
- MARIADB_USER=wordpress
|
||||
- MARIADB_PASSWORD=wordpress
|
||||
volumes:
|
||||
- mariadb-dev-data:/var/lib/mysql
|
||||
networks:
|
||||
- maple-dev
|
||||
healthcheck:
|
||||
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
restart: unless-stopped
|
||||
|
||||
# WordPress - Plugin development and testing
|
||||
# Access: http://localhost:8081
|
||||
# Plugin auto-mounted from: native/wordpress/maplepress-plugin
|
||||
# Debug logs: docker exec -it maple-wordpress-dev tail -f /var/www/html/wp-content/debug.log
|
||||
wordpress:
|
||||
image: wordpress:latest
|
||||
container_name: maple-wordpress-dev
|
||||
hostname: wordpress
|
||||
ports:
|
||||
- "8081:80"
|
||||
environment:
|
||||
- WORDPRESS_DB_HOST=mariadb:3306
|
||||
- WORDPRESS_DB_USER=wordpress
|
||||
- WORDPRESS_DB_PASSWORD=wordpress
|
||||
- WORDPRESS_DB_NAME=wordpress
|
||||
- WORDPRESS_DEBUG=1
|
||||
- WORDPRESS_CONFIG_EXTRA=
|
||||
define('WP_DEBUG', true);
|
||||
define('WP_DEBUG_LOG', true);
|
||||
define('WP_DEBUG_DISPLAY', false);
|
||||
volumes:
|
||||
- wordpress-dev-data:/var/www/html
|
||||
# MaplePress plugin - mounted read-only for live development
|
||||
- ../../../native/wordpress/maplepress-plugin:/var/www/html/wp-content/plugins/maplepress-plugin:ro
|
||||
networks:
|
||||
- maple-dev
|
||||
depends_on:
|
||||
mariadb:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:80/"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 60s
|
||||
restart: unless-stopped
|
||||
51
cloud/infrastructure/development/nginx/seaweedfs-cors.conf
Normal file
51
cloud/infrastructure/development/nginx/seaweedfs-cors.conf
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
server {
|
||||
listen 8334;
|
||||
server_name localhost;
|
||||
|
||||
# Map to dynamically set CORS origin based on request
|
||||
# This allows multiple localhost ports for development
|
||||
set $cors_origin "";
|
||||
if ($http_origin ~* "^http://localhost:(5173|5174|5175|3000|8080)$") {
|
||||
set $cors_origin $http_origin;
|
||||
}
|
||||
|
||||
# Proxy to SeaweedFS S3 endpoint
|
||||
location / {
|
||||
# Hide CORS headers from upstream SeaweedFS (to prevent duplicates)
|
||||
proxy_hide_header 'Access-Control-Allow-Origin';
|
||||
proxy_hide_header 'Access-Control-Allow-Methods';
|
||||
proxy_hide_header 'Access-Control-Allow-Headers';
|
||||
proxy_hide_header 'Access-Control-Expose-Headers';
|
||||
proxy_hide_header 'Access-Control-Max-Age';
|
||||
proxy_hide_header 'Access-Control-Allow-Credentials';
|
||||
|
||||
# CORS Headers for development - dynamically set based on request origin
|
||||
add_header 'Access-Control-Allow-Origin' $cors_origin always;
|
||||
add_header 'Access-Control-Allow-Methods' 'GET, PUT, POST, DELETE, HEAD, OPTIONS' always;
|
||||
add_header 'Access-Control-Allow-Headers' '*' always;
|
||||
add_header 'Access-Control-Expose-Headers' 'ETag, Content-Length, Content-Type' always;
|
||||
add_header 'Access-Control-Max-Age' '3600' always;
|
||||
|
||||
# Handle preflight requests
|
||||
if ($request_method = 'OPTIONS') {
|
||||
add_header 'Access-Control-Allow-Origin' $cors_origin always;
|
||||
add_header 'Access-Control-Allow-Methods' 'GET, PUT, POST, DELETE, HEAD, OPTIONS' always;
|
||||
add_header 'Access-Control-Allow-Headers' '*' always;
|
||||
add_header 'Access-Control-Max-Age' '3600' always;
|
||||
add_header 'Content-Type' 'text/plain; charset=utf-8' always;
|
||||
add_header 'Content-Length' '0' always;
|
||||
return 204;
|
||||
}
|
||||
|
||||
# Proxy to SeaweedFS
|
||||
proxy_pass http://seaweedfs:8333;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Important for large file uploads
|
||||
proxy_request_buffering off;
|
||||
client_max_body_size 1G;
|
||||
}
|
||||
}
|
||||
23
cloud/infrastructure/development/redis/redis.dev.conf
Normal file
23
cloud/infrastructure/development/redis/redis.dev.conf
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
# Maple Infrastructure - Redis Development Configuration
|
||||
|
||||
# Network
|
||||
bind 0.0.0.0
|
||||
port 6379
|
||||
protected-mode no
|
||||
|
||||
# Persistence
|
||||
save 900 1
|
||||
save 300 10
|
||||
save 60 10000
|
||||
appendonly yes
|
||||
appendfilename "appendonly.aof"
|
||||
|
||||
# Limits
|
||||
maxmemory 256mb
|
||||
maxmemory-policy allkeys-lru
|
||||
|
||||
# Logging
|
||||
loglevel notice
|
||||
|
||||
# Databases (default 16)
|
||||
databases 16
|
||||
Loading…
Add table
Add a link
Reference in a new issue