Bartlomiej Mika 480a2b557d Saved our architecture pattern.

2025-12-05 14:05:19 -05:00

29 KiB

Raw Blame History

Shared Infrastructure Pattern Guide

A comprehensive guide to implementing a shared development infrastructure for microservices

This document explains an architectural pattern where multiple backend services, frontends, and applications share a common local development infrastructure. Use this as a blueprint for setting up similar environments in your own projects.

📚 Table of Contents

What is the Shared Infrastructure Pattern?
Why Use This Pattern?
Architecture Overview
Directory Structure
Core Concepts
Step-by-Step Implementation
Connecting Applications
Development Workflow
Best Practices
Troubleshooting Common Issues

What is the Shared Infrastructure Pattern?

The Shared Infrastructure Pattern separates your development infrastructure (databases, cache, storage, etc.) from your application code. Instead of each project running its own database containers, all projects share a single set of infrastructure services.

Traditional Approach:

project-a/
  ├── docker-compose.yml  (includes database, cache, app)
  └── app code

project-b/
  ├── docker-compose.yml  (includes database, cache, app)
  └── app code

Problem: Each project starts its own infrastructure (slow, resource-heavy)

Shared Infrastructure Approach:

infrastructure/
  ├── docker-compose.dev.yml  (database, cache, etc.)
  └── task commands

project-a/
  ├── docker-compose.dev.yml  (app only)
  └── app code

project-b/
  ├── docker-compose.dev.yml  (app only)
  └── app code

Solution: Infrastructure starts once, apps restart quickly

Why Use This Pattern?

Benefits

✅ Faster Development Iterations

Infrastructure starts once (2-3 minutes for complex databases)
Apps restart in seconds
No waiting for databases during code changes

✅ Production-Like Environment

Mimics real microservices architecture
Services communicate via networks (not localhost)
Learn proper service separation

✅ Resource Efficiency

One database cluster instead of multiple
Shared cache, storage, and search services
Lower memory and CPU usage

✅ Consistency Across Team

All developers use identical infrastructure
Same configuration everywhere
Easier onboarding for new team members

✅ Simplified Management

Start/stop infrastructure independently
Easier to upgrade shared services
Centralized configuration

Trade-offs

⚠️ Slightly More Complex

One extra terminal/directory to manage
Initial setup more involved than single docker-compose
Requires understanding of Docker networks

Decision Point: Use this pattern when you have 2+ applications sharing infrastructure. For single apps, a monolithic docker-compose is simpler.

Architecture Overview

┌─────────────────────────────────────────────────────────┐
│                  Development Machine                     │
│                                                           │
│  ┌────────────────────────────────────────────────────┐ │
│  │         infrastructure/development/                 │ │
│  │  ┌──────────────────────────────────────────────┐  │ │
│  │  │  Docker Network: shared-dev-network          │  │ │
│  │  │                                                │  │ │
│  │  │  ┌──────────┐  ┌────────┐  ┌─────────────┐  │  │ │
│  │  │  │ Database │  │ Cache  │  │   Storage   │  │  │ │
│  │  │  │ (Postgres│  │(Redis) │  │   (S3/      │  │  │ │
│  │  │  │  MySQL,  │  │        │  │   MinIO)    │  │  │ │
│  │  │  │ Cassandra│  │        │  │             │  │  │ │
│  │  │  └──────────┘  └────────┘  └─────────────┘  │  │ │
│  │  │                                                │  │ │
│  │  │  ┌──────────┐  ┌────────┐  ┌─────────────┐  │  │ │
│  │  │  │  Search  │  │ Queue  │  │   Others    │  │  │ │
│  │  │  │(Elastic, │  │(Rabbit,│  │ (monitoring,│  │  │ │
│  │  │  │ Meili)   │  │ Kafka) │  │  mail, etc.)│  │  │ │
│  │  │  └──────────┘  └────────┘  └─────────────┘  │  │ │
│  │  └──────────────────────────────────────────────┘  │ │
│  └────────────────────────────────────────────────────┘ │
│                                                           │
│  ┌────────────────────────────────────────────────────┐ │
│  │  Applications (connect to shared network)          │ │
│  │                                                      │ │
│  │  ┌──────────────┐      ┌──────────────┐           │ │
│  │  │  Backend A   │      │  Backend B   │           │ │
│  │  │  (port 8000) │      │  (port 8001) │           │ │
│  │  └──────────────┘      └──────────────┘           │ │
│  │                                                      │ │
│  │  ┌──────────────┐      ┌──────────────┐           │ │
│  │  │ Frontend A   │      │ Frontend B   │           │ │
│  │  │ (npm dev)    │      │ (npm dev)    │           │ │
│  │  └──────────────┘      └──────────────┘           │ │
│  └────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────┘

Key Concepts:

Shared Docker Network
- All services join one network
- Services discover each other by hostname
- Example: Backend connects to database:5432 or cache:6379
Persistent Volumes
- Data survives container restarts
- Named volumes (e.g., myproject-db-dev, myproject-cache-dev)
- Only deleted when explicitly requested
Service Isolation
- Each application gets its own database/schema
- Each application gets its own cache namespace
- Logical separation, physical sharing

Directory Structure

your-project-root/
├── infrastructure/
│   ├── development/
│   │   ├── docker-compose.dev.yml     # All infrastructure services
│   │   ├── Taskfile.yml               # Commands (start, stop, etc.)
│   │   ├── README.md                  # Setup instructions
│   │   ├── .env                       # Infrastructure config (optional)
│   │   │
│   │   ├── database/
│   │   │   └── init-scripts/          # Database initialization
│   │   │       ├── 01-create-databases.sql
│   │   │       └── 02-create-users.sql
│   │   │
│   │   ├── cache/
│   │   │   └── redis.conf             # Cache configuration
│   │   │
│   │   └── nginx/                     # Reverse proxy configs
│   │       └── default.conf
│   │
│   └── production/                     # Production deployment
│       └── ... (out of scope)
│
├── services/                           # Backend services
│   ├── service-a/
│   │   ├── docker-compose.dev.yml     # App only (connects to infra)
│   │   ├── Taskfile.yml               # App commands
│   │   ├── .env.sample                # Environment template
│   │   ├── .env                       # Actual config (gitignored)
│   │   └── ... (app code)
│   │
│   └── service-b/
│       └── ... (similar structure)
│
├── web/                                # Frontend applications
│   ├── app-a/
│   │   ├── package.json               # npm run dev
│   │   ├── .env.local                 # Frontend config
│   │   └── ... (React/Vue/Svelte)
│   │
│   └── app-b/
│       └── ...
│
├── Taskfile.yml                        # Root commands (optional)
└── README.md                           # Project overview

Naming Flexibility: Use whatever naming suits your project:

backend/, api/, services/, cloud/
frontend/, web/, client/, ui/
infra/, infrastructure/, docker/

Core Concepts

1. Shared Docker Network

Infrastructure creates the network:

# infrastructure/development/docker-compose.dev.yml
networks:
  shared-dev-network:
    name: my-project-dev
    driver: bridge

Applications join the network:

# services/service-a/docker-compose.dev.yml
networks:
  shared-dev-network:
    external: true  # Important: marks it as pre-existing

services:
  app:
    networks:
      - shared-dev-network

2. Service Discovery by Hostname

Services use hostnames (not IP addresses) to find each other:

# infrastructure/development/docker-compose.dev.yml
services:
  database:
    hostname: database
    # or postgres, mysql, db, etc.

  cache:
    hostname: cache
    # or redis, memcached, etc.

In your application code:

# Database connection
DATABASE_HOST=database
DATABASE_PORT=5432

# Cache connection
CACHE_HOST=cache
CACHE_PORT=6379

3. Data Persistence with Named Volumes

# infrastructure/development/docker-compose.dev.yml
volumes:
  db-data:
    name: myproject-db-dev
  cache-data:
    name: myproject-cache-dev

services:
  database:
    volumes:
      - db-data:/var/lib/postgresql/data

  cache:
    volumes:
      - cache-data:/data

Data survives:

Container restarts
Code changes
Docker updates

Data deleted only when:

You explicitly run cleanup command
You delete volumes manually

4. Database Isolation Strategies

Option A: Separate Databases/Schemas

-- PostgreSQL/MySQL example
CREATE DATABASE app_a;
CREATE DATABASE app_b;

# service-a
environment:
  DATABASE_NAME: app_a

# service-b
environment:
  DATABASE_NAME: app_b

Option B: Schema/Keyspace Separation (Cassandra, MongoDB)

-- Cassandra example
CREATE KEYSPACE app_a WITH replication = {...};
CREATE KEYSPACE app_b WITH replication = {...};

Option C: Cache Database Numbers (Redis)

# Redis supports 16 databases (0-15)
# service-a
CACHE_DB=0

# service-b
CACHE_DB=1

5. Port Management

Infrastructure exposes ports for debugging:

services:
  database:
    ports:
      - "5432:5432"  # Access from host for debugging

  cache:
    ports:
      - "6379:6379"

Applications expose unique ports:

# service-a
ports:
  - "8000:8000"

# service-b
ports:
  - "8001:8000"  # Different host port, same container port

6. Health Checks

Ensure services are ready before applications connect:

services:
  database:
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s

  cache:
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 3

Step-by-Step Implementation

Phase 1: Create Infrastructure

Step 1.1: Create Directory Structure

mkdir -p infrastructure/development/{database,cache,storage}/init-scripts
cd infrastructure/development

Step 1.2: Create `docker-compose.dev.yml`

Start with minimal services (add more as needed):

version: "3.9"

networks:
  shared-dev-network:
    name: myproject-dev
    driver: bridge

volumes:
  db-data:
    name: myproject-db-dev
  cache-data:
    name: myproject-cache-dev

services:
  database:
    image: postgres:16  # or mysql:8, cassandra:5, etc.
    container_name: myproject-db-dev
    hostname: database
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: devpassword
      POSTGRES_USER: devuser
    volumes:
      - db-data:/var/lib/postgresql/data
      - ./database/init-scripts:/docker-entrypoint-initdb.d:ro
    networks:
      - shared-dev-network
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U devuser"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  cache:
    image: redis:7-alpine
    container_name: myproject-cache-dev
    hostname: cache
    ports:
      - "6379:6379"
    volumes:
      - cache-data:/data
    networks:
      - shared-dev-network
    command: redis-server --appendonly yes
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 3
    restart: unless-stopped

Add more services as needed:

Search engines (Elasticsearch, Meilisearch)
Message queues (RabbitMQ, Kafka)
Object storage (MinIO, SeaweedFS)
Monitoring (Prometheus, Grafana)

Step 1.3: Create Database Initialization Scripts

PostgreSQL example: database/init-scripts/01-create-databases.sql

-- Create databases for each application
CREATE DATABASE service_a;
CREATE DATABASE service_b;

-- Create users if needed
CREATE USER app_a_user WITH PASSWORD 'devpassword';
GRANT ALL PRIVILEGES ON DATABASE service_a TO app_a_user;

CREATE USER app_b_user WITH PASSWORD 'devpassword';
GRANT ALL PRIVILEGES ON DATABASE service_b TO app_b_user;

MySQL example:

CREATE DATABASE IF NOT EXISTS service_a;
CREATE DATABASE IF NOT EXISTS service_b;

CREATE USER 'app_a_user'@'%' IDENTIFIED BY 'devpassword';
GRANT ALL PRIVILEGES ON service_a.* TO 'app_a_user'@'%';

CREATE USER 'app_b_user'@'%' IDENTIFIED BY 'devpassword';
GRANT ALL PRIVILEGES ON service_b.* TO 'app_b_user'@'%';

FLUSH PRIVILEGES;

Step 1.4: Create `Taskfile.yml`

version: "3"

vars:
  DOCKER_COMPOSE_CMD:
    sh: |
      if command -v docker-compose >/dev/null 2>&1; then
        echo "docker-compose"
      elif docker compose version >/dev/null 2>&1; then
        echo "docker compose"
      else
        echo "docker-compose"
      fi

tasks:
  dev:start:
    desc: Start all infrastructure services
    cmds:
      - "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml up -d"
      - echo "⏳ Waiting for services..."
      - task: dev:wait
      - echo "✅ Infrastructure ready!"
      - task: dev:status

  dev:wait:
    desc: Wait for services to be healthy
    silent: true
    cmds:
      - |
        echo "Waiting for database..."
        for i in {1..30}; do
          if docker exec myproject-db-dev pg_isready -U devuser >/dev/null 2>&1; then
            echo "✅ Database ready"
            break
          fi
          sleep 2
        done
      - |
        echo "Waiting for cache..."
        for i in {1..10}; do
          if docker exec myproject-cache-dev redis-cli ping >/dev/null 2>&1; then
            echo "✅ Cache ready"
            break
          fi
          sleep 1
        done

  dev:status:
    desc: Show status of all services
    cmds:
      - docker ps --filter "name=myproject-" --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

  dev:stop:
    desc: Stop all services (keeps data)
    cmds:
      - "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml down"
      - echo "✅ Infrastructure stopped (data preserved)"

  dev:restart:
    desc: Restart all services
    cmds:
      - task: dev:stop
      - task: dev:start

  dev:clean:
    desc: Stop and remove all data (DESTRUCTIVE!)
    prompt: This will DELETE ALL DATA. Continue?
    cmds:
      - "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml down -v"
      - echo "✅ All data deleted"

  dev:logs:
    desc: View logs (usage task dev:logs -- database)
    cmds:
      - "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml logs -f {{.CLI_ARGS}}"

  # Database-specific commands
  db:shell:
    desc: Open database shell
    cmds:
      - docker exec -it myproject-db-dev psql -U devuser

  cache:shell:
    desc: Open cache shell
    cmds:
      - docker exec -it myproject-cache-dev redis-cli

Step 1.5: Create `README.md`

Document:

What services are included
Prerequisites (Docker, Task)
How to start/stop
Port mappings
Connection examples
Troubleshooting

Step 1.6: Test Infrastructure

# Start
task dev:start

# Verify
task dev:status
# All services should show "healthy"

# Test database
task db:shell
# \l  (list databases)
# \q  (quit)

# Test cache
task cache:shell
# PING
# exit

# Stop
task dev:stop

Phase 2: Connect First Application

Step 2.1: Create Application Structure

mkdir -p services/service-a
cd services/service-a

Step 2.2: Create Application `docker-compose.dev.yml`

networks:
  shared-dev-network:
    external: true  # Connect to infrastructure network

services:
  app:
    container_name: service-a-dev
    build:
      context: .
      dockerfile: ./Dockerfile.dev
    ports:
      - "8000:8000"
    env_file:
      - .env
    environment:
      # Database connection (use infrastructure hostname)
      DATABASE_HOST: database
      DATABASE_PORT: 5432
      DATABASE_NAME: service_a
      DATABASE_USER: app_a_user
      DATABASE_PASSWORD: devpassword

      # Cache connection
      CACHE_HOST: cache
      CACHE_PORT: 6379
      CACHE_DB: 0

      # Application settings
      APP_PORT: 8000
      APP_ENV: development

    volumes:
      - ./:/app  # Mount source for live reload
    networks:
      - shared-dev-network
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 5s
      retries: 3

Step 2.3: Create Application `Taskfile.yml`

version: "3"

env:
  COMPOSE_PROJECT_NAME: service-a

vars:
  DOCKER_COMPOSE_CMD:
    sh: |
      if command -v docker-compose >/dev/null 2>&1; then
        echo "docker-compose"
      elif docker compose version >/dev/null 2>&1; then
        echo "docker compose"
      else
        echo "docker-compose"
      fi

tasks:
  dev:
    desc: Start application (requires infrastructure running)
    deps: [check-infra]
    cmds:
      - "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml up --build"

  check-infra:
    desc: Verify infrastructure is running
    silent: true
    cmds:
      - |
        if ! docker network inspect myproject-dev >/dev/null 2>&1; then
          echo "❌ Infrastructure not running!"
          echo "Start with: cd ../../infrastructure/development && task dev:start"
          exit 1
        fi
        echo "✅ Infrastructure is running"

  dev:down:
    desc: Stop application
    cmds:
      - "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml down"

  dev:restart:
    desc: Quick restart
    cmds:
      - "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml restart"
      - echo "✅ Application restarted"

  dev:logs:
    desc: View application logs
    cmds:
      - "{{.DOCKER_COMPOSE_CMD}} -f docker-compose.dev.yml logs -f"

  dev:shell:
    desc: Open shell in container
    cmds:
      - docker exec -it service-a-dev sh

  test:
    desc: Run tests
    cmds:
      - # Your test command (e.g., go test ./..., npm test, pytest)

  lint:
    desc: Run linters
    cmds:
      - # Your lint command

Step 2.4: Create `.env.sample`

# Application Configuration
APP_ENV=development
APP_PORT=8000

# Database (configured in docker-compose.dev.yml)
# DATABASE_HOST=database
# DATABASE_NAME=service_a

# Cache (configured in docker-compose.dev.yml)
# CACHE_HOST=cache
# CACHE_DB=0

# Application Secrets (override in .env)
JWT_SECRET=change-in-production
API_KEY=your-api-key-here

Step 2.5: Test Application

# Ensure infrastructure is running
cd ../../infrastructure/development
task dev:status

# Start application
cd ../../services/service-a
task dev

# Test
curl http://localhost:8000/health

# Stop
task dev:down

Phase 3: Add More Applications

For each additional service:

Create new service directory
Copy and modify docker-compose.dev.yml
- Change container name
- Change port (8001, 8002, etc.)
- Change database name
- Change cache DB number (1, 2, etc.)
Copy and modify Taskfile.yml
Update infrastructure init scripts (add new database)

Phase 4: Add Frontend Applications

Frontends typically run outside Docker for faster hot-reload:

mkdir -p web/frontend-a
cd web/frontend-a

# Initialize (React example)
npm create vite@latest . -- --template react
npm install

Configure API connection: .env.local

VITE_API_URL=http://localhost:8000

Start:

npm run dev

Alternative: Run frontend in Docker (slower hot-reload)

# web/frontend-a/docker-compose.dev.yml
services:
  app:
    image: node:20-alpine
    working_dir: /app
    command: npm run dev
    ports:
      - "5173:5173"
    volumes:
      - ./:/app
    environment:
      - VITE_API_URL=http://localhost:8000

Connecting Applications

Service Discovery

Use hostnames (defined in infrastructure docker-compose):

# infrastructure: hostname set
services:
  database:
    hostname: database

  cache:
    hostname: cache

  storage:
    hostname: storage

# application: use hostnames
DATABASE_HOST=database
DATABASE_PORT=5432

CACHE_HOST=cache
CACHE_PORT=6379

STORAGE_ENDPOINT=http://storage:9000

Database Isolation

Per-application databases:

CREATE DATABASE app_a;
CREATE DATABASE app_b;

Per-application schemas:

CREATE SCHEMA app_a;
CREATE SCHEMA app_b;

Per-application cache DBs:

# Redis DB 0-15
CACHE_DB=0  # app-a
CACHE_DB=1  # app-b

Network Configuration

Infrastructure creates network:

networks:
  shared-dev-network:
    name: myproject-dev
    driver: bridge

services:
  database:
    networks:
      - shared-dev-network

Applications join network:

networks:
  shared-dev-network:
    external: true  # Pre-existing network

services:
  app:
    networks:
      - shared-dev-network

Development Workflow

Daily Workflow

# Morning: Start infrastructure (once)
cd infrastructure/development
task dev:start
# Takes 1-3 minutes

# Start service(s)
cd ../../services/service-a
task dev
# Takes 10-30 seconds

# Work on code, restart quickly as needed
task dev:restart
# Takes 5-10 seconds

# Start frontend (separate terminal)
cd ../../web/frontend-a
npm run dev
# Takes 5-10 seconds

# End of day (optional)
cd ../../infrastructure/development
task dev:stop

Adding Features

# Run database migrations
cd services/service-a
./migrate up

# Or use your migration tool
npm run migrate
python manage.py migrate

Debugging

# Infrastructure logs
cd infrastructure/development
task dev:logs -- database
task dev:logs -- cache

# Application logs
cd ../../services/service-a
task dev:logs

# Database shell
cd ../../infrastructure/development
task db:shell

# Cache shell
task cache:shell

# Application shell
cd ../../services/service-a
task dev:shell

Resetting Data

# Reset infrastructure (deletes all data)
cd infrastructure/development
task dev:clean
task dev:start

# Reset specific database
task db:shell
# DROP DATABASE service_a;
# CREATE DATABASE service_a;

Best Practices

1. Naming Conventions

Be consistent across your project:

Infrastructure:
- Network: {project}-dev
- Containers: {project}-{service}-dev
- Volumes: {project}-{service}-dev

Applications:
- Containers: {appname}-dev
- Ports: 8000, 8001, 8002, ...

2. Always Use Health Checks

healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost/health"]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 60s

3. Document Everything

Infrastructure README:

What services are included
How to start/stop
Port mappings
Connection examples
Troubleshooting

Application README:

Link to infrastructure setup
Application-specific setup
Environment variables
How to run tests

4. Use Environment Variables

Infrastructure: Minimal configuration (most settings in docker-compose)

Applications: Use .env files:

Keep .env.sample in version control
Add .env to .gitignore
Document all variables

5. Port Management

Reserve port ranges:

5432: PostgreSQL
3306: MySQL
9042: Cassandra
6379: Redis
9200: Elasticsearch
8000-8099: Backend services
5173-5199: Frontend dev servers

6. Resource Limits (Optional)

For limited dev machines:

services:
  database:
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G

7. Startup Dependencies

Use health conditions:

services:
  app:
    depends_on:
      database:
        condition: service_healthy

8. Cleanup Strategy

# Remove unused Docker resources
docker system prune

# Remove specific project volumes
docker volume rm myproject-db-dev myproject-cache-dev

Troubleshooting Common Issues

Issue 1: Network Not Found

Error:

ERROR: Network shared-dev-network declared as external, but could not be found

Solution:

# Start infrastructure first
cd infrastructure/development
task dev:start

Issue 2: Port Already in Use

Error:

Error: bind: address already in use

Solution:

# Find process using port
lsof -i :5432  # macOS/Linux
netstat -ano | findstr :5432  # Windows

# Kill process or change port in docker-compose.yml
ports:
  - "5433:5432"  # Use different host port

Issue 3: Service Unhealthy

Error:

service-name   Up 2 minutes (unhealthy)

Solution:

# Check logs
task dev:logs -- database

# Wait longer (some services are slow to start)

# Restart if needed
task dev:restart

Issue 4: Can't Connect to Database

Error:

Connection refused: database:5432

Solution:

Verify application is on shared network:

networks:
  shared-dev-network:
    external: true

services:
  app:
    networks:
      - shared-dev-network

Verify infrastructure is running:

docker network inspect myproject-dev

Issue 5: Data Persists After Container Restart

This is expected! Data in named volumes persists.

To reset:

task dev:clean  # Deletes volumes
task dev:start

Issue 6: Docker Compose Command Not Found

Error:

docker-compose: command not found

Solution:

The Taskfile auto-detects both docker-compose (v1) and docker compose (v2).

If both fail:

# Install Docker Desktop (includes docker-compose)
# Or check: docker compose version

Issue 7: Permission Denied on Linux

Error:

Permission denied: /var/lib/database

Solution:

# Add user to docker group
sudo usermod -aG docker $USER
# Log out and back in

# Or run with sudo (not recommended)

Summary

The Shared Infrastructure Pattern provides:

✅ Fast iterations - Restart apps in seconds, not minutes ✅ Production-like - Proper microservices architecture ✅ Resource efficient - One infrastructure, many apps ✅ Team consistency - Everyone uses same setup ✅ Easy maintenance - Centralized upgrades

When to use:

2+ applications sharing infrastructure
Team of 2+ developers
Microservices architecture
Learning distributed systems

When not to use:

Single application
Simple monolithic architecture
Very limited resources

Next Steps

✅ Understand the pattern - Re-read if needed
✅ Plan your services - What infrastructure do you need?
✅ Implement Phase 1 - Set up shared infrastructure
✅ Implement Phase 2 - Connect first application
✅ Test thoroughly - Ensure communication works
✅ Document - Write clear READMEs
✅ Add more services - Repeat for additional apps

Additional Resources

Docker Compose: https://docs.docker.com/compose/
Docker Networks: https://docs.docker.com/network/
Task (Taskfile): https://taskfile.dev/
Docker Best Practices: https://docs.docker.com/develop/dev-best-practices/

Questions or improvements? This pattern is used in production by many teams. Adapt it to your needs!

29 KiB Raw Blame History

Shared Infrastructure Pattern Guide

📚 Table of Contents

What is the Shared Infrastructure Pattern?

Why Use This Pattern?

Benefits

Trade-offs

Architecture Overview

Key Concepts:

Directory Structure

Core Concepts

1. Shared Docker Network

2. Service Discovery by Hostname

3. Data Persistence with Named Volumes

4. Database Isolation Strategies

5. Port Management

6. Health Checks

Step-by-Step Implementation

Phase 1: Create Infrastructure

Step 1.1: Create Directory Structure

Step 1.2: Create docker-compose.dev.yml

Step 1.3: Create Database Initialization Scripts

Step 1.4: Create Taskfile.yml

Step 1.5: Create README.md

Step 1.6: Test Infrastructure

Phase 2: Connect First Application

Step 2.1: Create Application Structure

Step 2.2: Create Application docker-compose.dev.yml

Step 2.3: Create Application Taskfile.yml

Step 2.4: Create .env.sample

Step 2.5: Test Application

Phase 3: Add More Applications

Phase 4: Add Frontend Applications

Connecting Applications

Service Discovery

Database Isolation

Network Configuration

Development Workflow

Daily Workflow

Adding Features

Debugging

Resetting Data

Best Practices

1. Naming Conventions

2. Always Use Health Checks

3. Document Everything

4. Use Environment Variables

5. Port Management

6. Resource Limits (Optional)

7. Startup Dependencies

8. Cleanup Strategy

Troubleshooting Common Issues

Issue 1: Network Not Found

Issue 2: Port Already in Use

Issue 3: Service Unhealthy

Issue 4: Can't Connect to Database

Issue 5: Data Persists After Container Restart

Issue 6: Docker Compose Command Not Found

Issue 7: Permission Denied on Linux

Summary

Next Steps

Additional Resources

29 KiB

Raw Blame History

Step 1.2: Create `docker-compose.dev.yml`

Step 1.4: Create `Taskfile.yml`

Step 1.5: Create `README.md`

Step 2.2: Create Application `docker-compose.dev.yml`

Step 2.3: Create Application `Taskfile.yml`

Step 2.4: Create `.env.sample`