2823 lines
84 KiB
Markdown
2823 lines
84 KiB
Markdown
# MaplePress Backend - Developer Guide
|
|
|
|
**Last Updated**: 2025-10-30
|
|
|
|
This guide provides everything you need to understand and contribute to the MaplePress Backend codebase.
|
|
|
|
---
|
|
|
|
## Table of Contents
|
|
|
|
1. [Overview](#overview)
|
|
2. [Architecture Overview](#architecture-overview)
|
|
3. [Module Organization](#module-organization)
|
|
4. [Key Architectural Decisions](#key-architectural-decisions)
|
|
5. [Authentication & Authorization](#authentication--authorization)
|
|
6. [Multi-Tenancy Implementation](#multi-tenancy-implementation)
|
|
7. [Working with Cassandra](#working-with-cassandra)
|
|
8. [Meilisearch Integration](#meilisearch-integration)
|
|
9. [Usage-Based Billing](#usage-based-billing)
|
|
10. [Scheduled Jobs](#scheduled-jobs)
|
|
11. [Rate Limiting Architecture](#rate-limiting-architecture)
|
|
12. [Adding New Features](#adding-new-features)
|
|
13. [Code Patterns & Conventions](#code-patterns--conventions)
|
|
14. [Testing Guidelines](#testing-guidelines)
|
|
15. [Common Pitfalls](#common-pitfalls)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
MaplePress Backend is a **multi-tenant SaaS platform** built with Go that provides cloud-powered services for WordPress sites. The primary feature is **cloud-based full-text search** using Meilisearch, with future expansion planned for file uploads, metrics, and analytics.
|
|
|
|
### Key Features
|
|
|
|
- **WordPress Plugin Integration** - API key authentication (Stripe-style) for WordPress plugins
|
|
- **Full-Text Search** - Meilisearch-powered search with per-site indexes
|
|
- **Multi-Tenant Architecture** - Shared tables with tenant isolation via partition keys
|
|
- **Usage-Based Billing** - Track all usage for billing (no quotas or limits)
|
|
- **Rate Limiting** - Generous anti-abuse limits (10K req/hour per API key)
|
|
- **Focused Use Cases** - Single-responsibility use cases for composable workflows
|
|
- **Clean Architecture** - Clear layer separation with dependency inversion
|
|
|
|
### Technology Stack
|
|
|
|
| Category | Technology | Purpose |
|
|
|----------|-----------|---------|
|
|
| **Language** | Go 1.24.4 | Backend language |
|
|
| **DI Framework** | Google Wire | Compile-time dependency injection |
|
|
| **CLI Framework** | Cobra | Command-line interface |
|
|
| **Database** | Cassandra 3.11 | Primary data store (3-node cluster) |
|
|
| **Cache** | Redis | Session storage, distributed locks |
|
|
| **Search** | Meilisearch | Full-text search engine |
|
|
| **Object Storage** | AWS S3 / S3-compatible | File storage (optional) |
|
|
| **Email** | Mailgun | Transactional emails |
|
|
| **Logger** | Uber Zap | Structured logging |
|
|
| **HTTP Router** | net/http (stdlib) | HTTP server (Go 1.22+) |
|
|
| **JWT** | golang-jwt/jwt v5 | JSON Web Tokens |
|
|
| **Password** | golang.org/x/crypto | Argon2id hashing |
|
|
| **Migrations** | golang-migrate/migrate v4 | Database migrations |
|
|
| **Cron** | robfig/cron v3 | Scheduled jobs |
|
|
| **UUID** | google/uuid, gocql UUID | Unique identifiers |
|
|
|
|
---
|
|
|
|
## Architecture Overview
|
|
|
|
MaplePress follows **Clean Architecture** with 5 distinct layers:
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────┐
|
|
│ Interface Layer (HTTP Handlers, DTOs, Middleware) │
|
|
│ internal/interface/http/ │
|
|
└──────────────────────┬──────────────────────────────┘
|
|
│
|
|
┌──────────────────────▼──────────────────────────────┐
|
|
│ Service Layer (Use Case Orchestration) │
|
|
│ internal/service/ │
|
|
└──────────────────────┬──────────────────────────────┘
|
|
│
|
|
┌──────────────────────▼──────────────────────────────┐
|
|
│ Use Case Layer (Focused Business Logic) │
|
|
│ internal/usecase/ │
|
|
└──────────────────────┬──────────────────────────────┘
|
|
│
|
|
┌──────────────────────▼──────────────────────────────┐
|
|
│ Repository Layer (Data Access) │
|
|
│ internal/repo/, internal/repository/ │
|
|
│ ├── models/ (Explicit Cassandra Table Models) │
|
|
└──────────────────────┬──────────────────────────────┘
|
|
│
|
|
┌──────────────────────▼──────────────────────────────┐
|
|
│ Domain Layer (Entities & Interfaces) │
|
|
│ internal/domain/ │
|
|
└─────────────────────────────────────────────────────┘
|
|
|
|
┌─────────────────────────────────────────────────────┐
|
|
│ Infrastructure (pkg/) │
|
|
│ ├── logger/ (Zap logger) │
|
|
│ ├── storage/ (Database, Cache, S3) │
|
|
│ ├── search/ (Meilisearch client) │
|
|
│ ├── security/ (JWT, API keys, passwords) │
|
|
│ ├── cache/ (Two-tier caching) │
|
|
│ ├── distributedmutex/ (Redis-based locks) │
|
|
│ ├── emailer/ (Mailgun) │
|
|
│ ├── httperror/ (HTTP error handling) │
|
|
│ └── httpresponse/ (HTTP response helpers) │
|
|
└─────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Directory Structure
|
|
|
|
```
|
|
maplepress-backend/
|
|
├── app/ # Dependency injection (Wire)
|
|
├── cmd/ # CLI commands (Cobra)
|
|
│ ├── daemon/ # Start HTTP server
|
|
│ ├── migrate/ # Database migrations
|
|
│ └── version/ # Version command
|
|
├── config/ # Configuration management
|
|
│ └── constants/ # Constants (session keys, etc.)
|
|
├── internal/ # Core application code
|
|
│ ├── domain/ # Domain entities & repository interfaces
|
|
│ ├── repo/ # Repository implementations (new pattern)
|
|
│ ├── repository/ # Repository implementations (old pattern)
|
|
│ ├── usecase/ # Focused use cases (single responsibility)
|
|
│ ├── service/ # Orchestration services
|
|
│ ├── interface/http/ # HTTP handlers, DTOs, middleware
|
|
│ ├── http/middleware/ # Additional middleware
|
|
│ └── scheduler/ # Cron jobs (quota reset)
|
|
├── pkg/ # Shared packages
|
|
│ ├── cache/ # Two-tier caching (Redis + Cassandra)
|
|
│ ├── distributedmutex/ # Distributed locks (Redis)
|
|
│ ├── emailer/ # Mailgun email service
|
|
│ ├── httperror/ # HTTP error utilities
|
|
│ ├── httpresponse/ # HTTP response helpers
|
|
│ ├── logger/ # Zap structured logging
|
|
│ ├── search/ # Meilisearch client
|
|
│ ├── security/ # JWT, API keys, password hashing
|
|
│ └── storage/ # Database, cache, S3 storage
|
|
├── migrations/ # Cassandra CQL migrations
|
|
└── static/ # Static files (blacklists, etc.)
|
|
```
|
|
|
|
### Dependency Rule
|
|
|
|
**Dependencies point INWARD**: Interface → Service → Use Case → Repository → Domain
|
|
|
|
- Outer layers depend on inner layers
|
|
- Inner layers never know about outer layers
|
|
- Domain layer has NO dependencies on other layers
|
|
- Repository interfaces defined in domain layer
|
|
- Implementations in repository/repo layer
|
|
|
|
### Layer Responsibilities
|
|
|
|
**Domain Layer** (`internal/domain/`)
|
|
- Pure business entities
|
|
- Repository interfaces (contracts)
|
|
- Domain errors
|
|
- Business validation logic
|
|
- No external dependencies
|
|
|
|
**Repository Layer** (`internal/repo/`, `internal/repository/`)
|
|
- Data access implementations
|
|
- Cassandra table models
|
|
- Query implementations
|
|
- Batched writes for consistency
|
|
- Two patterns: old (repository/) and new (repo/)
|
|
|
|
**Use Case Layer** (`internal/usecase/`)
|
|
- **Highly focused**, single-responsibility operations
|
|
- Composable building blocks
|
|
- Input/output structs (IDOs)
|
|
- Business orchestration at operation level
|
|
- Example: ValidatePlanTierUseCase, GenerateAPIKeyUseCase
|
|
|
|
**Service Layer** (`internal/service/`)
|
|
- Orchestrates multiple use cases into workflows
|
|
- Transaction boundaries
|
|
- Cross-cutting concerns
|
|
- Example: SyncPagesService orchestrates 7+ use cases
|
|
|
|
**Interface Layer** (`internal/interface/http/`)
|
|
- HTTP handlers
|
|
- DTOs (Data Transfer Objects)
|
|
- Middleware (JWT, API key, tenant extraction)
|
|
- Request/response transformation
|
|
- HTTP routing
|
|
|
|
---
|
|
|
|
## Module Organization
|
|
|
|
MaplePress is organized into domain modules, each with its own entities, repositories, use cases, and services.
|
|
|
|
### Core Modules
|
|
|
|
#### **Tenant Module** (`internal/domain/tenant/`)
|
|
**Purpose:** Top-level organization/customer entity in multi-tenant architecture
|
|
|
|
**Entity:** `Tenant`
|
|
- Name, Slug, Status (active/inactive/suspended)
|
|
- Root entity - not owned by any other tenant
|
|
|
|
**Repository:** `TenantRepository`
|
|
- Create, GetByID, GetBySlug, Update, Delete
|
|
- No tenant isolation (tenants are root entities)
|
|
|
|
**Database Tables:**
|
|
- `tenants_by_id` - Primary lookup
|
|
- `tenants_by_slug` - Unique slug lookup
|
|
- `tenants_by_status` - Status filtering
|
|
|
|
#### **User Module** (`internal/domain/user/`)
|
|
**Purpose:** User accounts belonging to tenants
|
|
|
|
**Entity:** `User`
|
|
- Email, Name, PasswordHash, Role, TenantID
|
|
- Every user belongs to exactly one tenant
|
|
- Role-based access (admin, user)
|
|
|
|
**Repository:** `UserRepository`
|
|
- CRUD operations with tenant isolation
|
|
- GetByEmail for authentication
|
|
- Argon2id password hashing
|
|
|
|
**Database Tables:**
|
|
- `users_by_id` - Primary lookup (partition: tenant_id, id)
|
|
- `users_by_email` - Email lookup
|
|
- `users_by_date` - List by creation date
|
|
|
|
#### **Site Module** (`internal/domain/site/`)
|
|
**Purpose:** WordPress sites registered in the system
|
|
|
|
**Entity:** `Site` - Comprehensive site management with usage tracking and authentication
|
|
|
|
**Fields:**
|
|
- **Identity:** ID, TenantID, Domain, SiteURL
|
|
- **Authentication:** APIKeyHash, APIKeyPrefix, APIKeyLastFour
|
|
- **Status:** Status (pending/active/inactive/suspended/archived), IsVerified, VerificationToken
|
|
- **Search:** SearchIndexName, TotalPagesIndexed, LastIndexedAt
|
|
- **Usage Tracking (for billing):**
|
|
- StorageUsedBytes - Cumulative storage consumption
|
|
- SearchRequestsCount - Monthly search API calls (resets monthly)
|
|
- MonthlyPagesIndexed - Pages indexed this month (resets monthly)
|
|
- LastResetAt - Billing cycle reset timestamp
|
|
|
|
**Repository:** `SiteRepository` (pattern in `internal/repo/`)
|
|
- Multi-table Cassandra pattern (4 tables for different access patterns)
|
|
- Batched writes for consistency
|
|
- Usage tracking updates
|
|
|
|
**Database Tables:**
|
|
1. `sites_by_id` - Primary table (partition: tenant_id, clustering: id)
|
|
2. `sites_by_tenant` - List view (partition: tenant_id, clustering: created_at)
|
|
3. `sites_by_domain` - Domain uniqueness (partition: domain)
|
|
4. `sites_by_apikey` - Fast authentication (partition: api_key_hash)
|
|
|
|
**Usage-Based Billing Model:**
|
|
- ✅ No plan tiers - All sites have same feature access
|
|
- ✅ No quota limits - Services never reject due to usage
|
|
- ✅ Usage tracking only - Track consumption for billing
|
|
- ✅ Monthly resets - Counters reset for billing cycles
|
|
- ✅ Rate limiting - Anti-abuse only (10K requests/hour per API key)
|
|
|
|
#### **Page Module** (`internal/domain/page/`)
|
|
**Purpose:** WordPress pages/posts indexed in the system
|
|
|
|
**Entity:** `Page`
|
|
- SiteID, PageID (WordPress ID), TenantID
|
|
- Title, Content (HTML stripped), Excerpt, URL
|
|
- Status (publish/draft/trash), PostType (page/post), Author
|
|
- PublishedAt, ModifiedAt, IndexedAt
|
|
- MeilisearchDocID (for search integration)
|
|
|
|
**Repository:** `PageRepository` (new pattern in `internal/repo/page/`)
|
|
- Page CRUD operations
|
|
- Batch operations for sync
|
|
|
|
**Database Table:**
|
|
- `pages_by_site` (partition: site_id, clustering: page_id)
|
|
|
|
#### **Session Module** (`internal/domain/session/`)
|
|
**Purpose:** User authentication sessions
|
|
|
|
**Entity:** `Session`
|
|
- SessionID, UserID, UserUUID, UserEmail, UserName, UserRole, TenantID, ExpiresAt
|
|
- JWT-based session management
|
|
- Tenant context for isolation
|
|
|
|
**Storage:** Redis cache (not persistent in Cassandra)
|
|
- TTL: 60 minutes (configurable)
|
|
- Auto-expiration
|
|
|
|
### HTTP Routes by Module
|
|
|
|
**Public Routes (no auth):**
|
|
```
|
|
GET /health # Health check
|
|
POST /api/v1/register # User registration
|
|
POST /api/v1/login # User login
|
|
```
|
|
|
|
**Authenticated Routes (JWT auth):**
|
|
```
|
|
POST /api/v1/hello # Test endpoint
|
|
GET /api/v1/me # Get current user
|
|
|
|
# Tenant management
|
|
POST /api/v1/tenants # Create tenant
|
|
GET /api/v1/tenants/{id} # Get tenant by ID
|
|
GET /api/v1/tenants/slug/{slug} # Get tenant by slug
|
|
```
|
|
|
|
**Tenant-Scoped Routes (JWT + Tenant context):**
|
|
```
|
|
# User management
|
|
POST /api/v1/users # Create user
|
|
GET /api/v1/users/{id} # Get user
|
|
|
|
# Site management
|
|
POST /api/v1/sites # Create site
|
|
GET /api/v1/sites # List sites
|
|
GET /api/v1/sites/{id} # Get site
|
|
DELETE /api/v1/sites/{id} # Delete site
|
|
POST /api/v1/sites/{id}/rotate-api-key # Rotate API key
|
|
```
|
|
|
|
**WordPress Plugin API (API Key auth):**
|
|
```
|
|
# Site status
|
|
GET /api/v1/plugin/status # Get site status & quotas
|
|
|
|
# Page sync
|
|
POST /api/v1/plugin/pages/sync # Sync pages to search
|
|
GET /api/v1/plugin/pages/status # Get sync status
|
|
GET /api/v1/plugin/pages/{page_id} # Get page by ID
|
|
|
|
# Page search
|
|
POST /api/v1/plugin/pages/search # Search pages
|
|
|
|
# Page deletion
|
|
DELETE /api/v1/plugin/pages # Delete specific pages
|
|
DELETE /api/v1/plugin/pages/all # Delete all pages
|
|
```
|
|
|
|
---
|
|
|
|
## Key Architectural Decisions
|
|
|
|
### ADR-001: Explicit Cassandra Table Models ✅
|
|
|
|
**Decision**: Use separate Go structs for each Cassandra table.
|
|
|
|
**Why?**
|
|
- Makes query-first data modeling visible in code
|
|
- Self-documenting (struct names explain purpose)
|
|
- Type-safe with compile-time checking
|
|
- Easy to maintain and refactor
|
|
|
|
**Example**:
|
|
```go
|
|
// Clear which table we're using
|
|
var userByID models.UserByID // → users_by_id table
|
|
var userByEmail models.UserByEmail // → users_by_email table
|
|
var userByDate models.UserByDate // → users_by_date table
|
|
```
|
|
|
|
**Structure**:
|
|
```
|
|
internal/repository/user/
|
|
├── models/
|
|
│ ├── user_by_id.go # UserByID struct → users_by_id table
|
|
│ ├── user_by_email.go # UserByEmail struct → users_by_email table
|
|
│ └── user_by_date.go # UserByDate struct → users_by_date table
|
|
├── impl.go # Repository struct
|
|
├── create.go # Create operations
|
|
├── get.go # Get operations
|
|
├── update.go # Update operations
|
|
├── delete.go # Delete operations
|
|
└── schema.cql # Cassandra schema
|
|
```
|
|
|
|
### ADR-002: Multi-Tenancy with Shared Tables ✅
|
|
|
|
**Decision**: Shared tables with `tenant_id` in partition keys (Option 3A).
|
|
|
|
**Why?**
|
|
- Scales to 10,000+ tenants
|
|
- Cost-effective ($0.10-1/tenant/month vs $500+/tenant/month for dedicated clusters)
|
|
- Simple operations (one schema, one migration)
|
|
- Industry-proven (Slack, GitHub, Stripe use this)
|
|
- Partition key ensures physical isolation
|
|
|
|
**Implementation**:
|
|
```cql
|
|
-- tenant_id is part of the partition key
|
|
CREATE TABLE users_by_id (
|
|
tenant_id UUID, -- Multi-tenant isolation
|
|
id UUID,
|
|
email TEXT,
|
|
name TEXT,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP,
|
|
PRIMARY KEY ((tenant_id, id)) -- Composite partition key
|
|
);
|
|
```
|
|
|
|
**Security**: All queries MUST include `tenant_id` - cannot query without it (would require table scan, which fails).
|
|
|
|
### ADR-003: Wire for Dependency Injection ✅
|
|
|
|
**Decision**: Use Google Wire for compile-time dependency injection.
|
|
|
|
**Why?**
|
|
- No runtime reflection overhead
|
|
- Errors caught at compile time
|
|
- Easy to debug (generated code is readable)
|
|
- Clear dependency graph
|
|
|
|
**Location**: `app/wire.go` (not in main package to avoid import cycles)
|
|
|
|
### ADR-004: Cobra for CLI ✅
|
|
|
|
**Decision**: Use Cobra for command-line interface.
|
|
|
|
**Commands**:
|
|
- `maplepress-backend daemon` - Start server
|
|
- `maplepress-backend version` - Show version
|
|
|
|
### ADR-005: DTOs in Separate Folder ✅
|
|
|
|
**Decision**: DTOs (Data Transfer Objects) live in `internal/interface/http/dto/[entity]/`
|
|
|
|
**Why?**
|
|
- API contracts are first-class citizens
|
|
- Clear separation from internal data structures
|
|
- Easy to version and evolve
|
|
|
|
### ADR-006: Use Case I/O = IDOs ✅
|
|
|
|
**Decision**: Use case input/output structs serve as IDOs (Internal Data Objects).
|
|
|
|
**Why?**
|
|
- Avoids unnecessary abstraction
|
|
- Use cases already define clear contracts
|
|
- Simpler codebase
|
|
|
|
### ADR-007: Focused Use Cases ✅
|
|
|
|
**Decision**: Break use cases into highly focused, single-responsibility operations.
|
|
|
|
**Why?**
|
|
- Each use case has one clear purpose
|
|
- Easy to test in isolation
|
|
- Composable building blocks
|
|
- Clear dependencies
|
|
- Service layer orchestrates multiple use cases
|
|
|
|
**Example:**
|
|
```go
|
|
// Traditional (monolithic):
|
|
CreateSiteUseCase // Does everything
|
|
|
|
// Refactored (focused):
|
|
ValidatePlanTierUseCase
|
|
ValidateDomainUseCase
|
|
GenerateAPIKeyUseCase
|
|
GenerateVerificationTokenUseCase
|
|
CreateSiteEntityUseCase
|
|
SaveSiteToRepoUseCase
|
|
```
|
|
|
|
### ADR-008: Dual Repository Pattern ✅
|
|
|
|
**Decision**: Coexist old (repository/) and new (repo/) repository patterns during migration.
|
|
|
|
**Why?**
|
|
- Allows gradual migration to simplified pattern
|
|
- New pattern (`repo/`) used for Site and Page modules
|
|
- Old pattern (`repository/`) used for User and Tenant modules
|
|
- Shows evolutionary refinement
|
|
|
|
---
|
|
|
|
## Authentication & Authorization
|
|
|
|
MaplePress uses **dual authentication**: JWT for users and API keys for WordPress plugins.
|
|
|
|
### User Authentication (JWT)
|
|
|
|
**Registration Flow:**
|
|
1. Validate input (email, password, tenant name/slug)
|
|
2. Check tenant slug uniqueness
|
|
3. Hash password (Argon2id with secure defaults)
|
|
4. Create tenant entity
|
|
5. Create user entity
|
|
6. Save both to database
|
|
7. Create session + generate JWT
|
|
8. Return JWT token
|
|
|
|
**Login Flow:**
|
|
1. Get user by email
|
|
2. Verify password (Argon2id)
|
|
3. Create session (Redis, 60min TTL)
|
|
4. Generate JWT token
|
|
5. Return token + user profile
|
|
|
|
**JWT Claims:**
|
|
- SessionID (UUID)
|
|
- Issued at, Expires at
|
|
- Secret: `APP_JWT_SECRET` (configurable)
|
|
|
|
**Middleware:** `JWTMiddleware`
|
|
- Validates JWT tokens (`Authorization: JWT <token>`)
|
|
- Populates context: `SessionID`, `UserID`, `UserEmail`, `UserRole`, `TenantID`
|
|
- Used for dashboard/admin routes
|
|
|
|
### WordPress Plugin Authentication (API Key)
|
|
|
|
**API Key Format:**
|
|
- **Production:** `live_sk_` + 40 random chars
|
|
- **Development:** `test_sk_` + 40 random chars
|
|
- **Total length:** 48 characters
|
|
|
|
**Generation Process:**
|
|
1. Generate 30 random bytes
|
|
2. Encode to base64url
|
|
3. Clean special chars, trim to 40 chars
|
|
4. Prefix with `live_sk_` or `test_sk_`
|
|
|
|
**Storage:**
|
|
- **Hash:** SHA-256 (stored in Cassandra `sites_by_apikey` table)
|
|
- **Display:** Prefix (first 13 chars) + Last 4 chars (e.g., `live_sk_abc...xyz1`)
|
|
- **Full key:** Shown only once at creation (never retrievable)
|
|
|
|
**Authentication Flow:**
|
|
1. Extract API key from `Authorization: Bearer` header
|
|
2. Hash API key (SHA-256)
|
|
3. Query `sites_by_apikey` table by hash
|
|
4. Validate site status (active/pending allowed)
|
|
5. Populate context with site details
|
|
|
|
**Middleware:** `APIKeyMiddleware`
|
|
- Validates API keys
|
|
- Populates context: `SiteID`, `SiteTenantID`, `SiteDomain`, `SitePlanTier`
|
|
- Used for WordPress plugin routes
|
|
|
|
**API Key Rotation:**
|
|
- `/api/v1/sites/{id}/rotate-api-key` endpoint
|
|
- Generates new API key
|
|
- Old key immediately invalid
|
|
- Return new key (shown only once)
|
|
|
|
---
|
|
|
|
## Meilisearch Integration
|
|
|
|
MaplePress uses **Meilisearch** for fast, typo-tolerant full-text search of WordPress content.
|
|
|
|
### Architecture
|
|
|
|
**Index Pattern:** One index per WordPress site
|
|
- Index name: `site_{site_id}` (e.g., `site_123e4567-e89b-12d3-a456-426614174000`)
|
|
- Isolated search data per site
|
|
- Independent index configuration
|
|
|
|
**Search Document Structure:**
|
|
```go
|
|
type PageDocument struct {
|
|
ID string `json:"id"` // Meilisearch document ID
|
|
SiteID string `json:"site_id"` // Site UUID
|
|
TenantID string `json:"tenant_id"` // Tenant UUID
|
|
PageID int64 `json:"page_id"` // WordPress page ID
|
|
Title string `json:"title"` // Page title
|
|
Content string `json:"content"` // HTML-stripped content
|
|
Excerpt string `json:"excerpt"` // Page excerpt
|
|
URL string `json:"url"` // Page URL
|
|
Status string `json:"status"` // publish/draft/trash
|
|
PostType string `json:"post_type"` // page/post
|
|
Author string `json:"author"` // Author name
|
|
PublishedAt int64 `json:"published_at"` // Unix timestamp
|
|
ModifiedAt int64 `json:"modified_at"` // Unix timestamp
|
|
}
|
|
```
|
|
|
|
### Page Sync Workflow
|
|
|
|
**Endpoint:** `POST /api/v1/plugin/pages/sync`
|
|
|
|
**Process:**
|
|
1. Authenticate API key
|
|
2. Validate site status and quotas
|
|
3. Check monthly indexing quota
|
|
4. Ensure Meilisearch index exists
|
|
5. For each page:
|
|
- Strip HTML from content
|
|
- Create page entity
|
|
- Upsert to Cassandra `pages_by_site` table
|
|
- Add to bulk index batch
|
|
6. Bulk index documents to Meilisearch
|
|
7. Update site quotas (pages indexed, last indexed timestamp)
|
|
8. Return sync summary
|
|
|
|
**Quota Enforcement:**
|
|
- Monthly indexing quota checked before sync
|
|
- Exceeding quota returns 403 Forbidden
|
|
- Quota resets monthly via cron job
|
|
|
|
### Search Workflow
|
|
|
|
**Endpoint:** `POST /api/v1/plugin/pages/search`
|
|
|
|
**Process:**
|
|
1. Authenticate API key
|
|
2. Validate site status and search quota
|
|
3. Check monthly search quota
|
|
4. Execute Meilisearch query
|
|
5. Increment search request counter
|
|
6. Return search results
|
|
|
|
**Search Features:**
|
|
- Typo tolerance (configurable)
|
|
- Faceted search
|
|
- Filtering by status, post type, author
|
|
- Custom ranking rules
|
|
- Pagination support
|
|
|
|
**Quota Enforcement:**
|
|
- Monthly search quota checked before search
|
|
- Exceeding quota returns 403 Forbidden
|
|
- Quota resets monthly via cron job
|
|
|
|
### Index Management
|
|
|
|
**Index Creation:**
|
|
- Automatic on first page sync
|
|
- Configured with searchable attributes: title, content, excerpt
|
|
- Filterable attributes: status, post_type, author
|
|
- Sortable attributes: published_at, modified_at
|
|
|
|
**Index Deletion:**
|
|
- When site is deleted
|
|
- Cascades to all indexed pages
|
|
|
|
---
|
|
|
|
## Usage-Based Billing
|
|
|
|
MaplePress uses a **usage-based billing model** with no quota limits or plan tiers. All usage is tracked for billing purposes.
|
|
|
|
### Usage Tracking (For Billing)
|
|
|
|
#### **Cumulative Metrics (Never Reset)**
|
|
- **Storage:** `StorageUsedBytes` - Total bytes stored across all pages
|
|
- **Total Pages:** `TotalPagesIndexed` - All-time page count
|
|
|
|
#### **Monthly Metrics (Reset Monthly for Billing Cycles)**
|
|
- **Searches:** `SearchRequestsCount` - Search API requests this month
|
|
- **Indexing:** `MonthlyPagesIndexed` - Pages indexed this month
|
|
- **Reset Tracking:** `LastResetAt` - When the monthly counters were last reset
|
|
|
|
### No Quota Enforcement
|
|
|
|
**✅ Page Sync - Always Allowed:**
|
|
```go
|
|
// ValidateSiteUseCase - No quota checks
|
|
site, err := uc.siteRepo.GetByID(ctx, tenantID, siteID)
|
|
if site.RequiresVerification() && !site.IsVerified {
|
|
return nil, domainsite.ErrSiteNotVerified
|
|
}
|
|
// Process pages without limits
|
|
```
|
|
|
|
**✅ Search - Always Allowed:**
|
|
```go
|
|
// Search service - No quota checks
|
|
result, err := uc.searchClient.Search(siteID.String(), searchReq)
|
|
// Always increment usage counter for billing
|
|
site.IncrementSearchCount()
|
|
uc.siteRepo.UpdateUsage(ctx, site)
|
|
```
|
|
|
|
### Usage Updates
|
|
|
|
**Simple Atomic Updates:**
|
|
```go
|
|
// Update usage tracking (no locks needed for counters)
|
|
site.MonthlyPagesIndexed += pagesIndexed
|
|
site.TotalPagesIndexed += pagesIndexed
|
|
site.SearchRequestsCount += 1
|
|
|
|
// Save to database
|
|
repo.UpdateUsage(ctx, site)
|
|
```
|
|
|
|
**Optimized Usage Tracking:**
|
|
- Use `UpdateUsageUseCase` for usage-only updates
|
|
- Avoids full site entity update
|
|
- Batched write to all 4 site tables
|
|
|
|
### Rate Limiting (Anti-Abuse Only)
|
|
|
|
MaplePress has generous rate limits to prevent abuse, not to enforce quotas:
|
|
|
|
**Plugin API Endpoints:**
|
|
- **Limit:** 10,000 requests/hour per API key
|
|
- **Purpose:** Anti-abuse only (prevent infinite loops, bugs)
|
|
- **Supports:** High-volume WordPress sites (240K requests/day)
|
|
- **Middleware:** `RateLimitMiddlewares` in `internal/http/middleware/`
|
|
|
|
---
|
|
|
|
## Scheduled Jobs
|
|
|
|
MaplePress uses **robfig/cron v3** for scheduled background tasks.
|
|
|
|
### Monthly Usage Reset Scheduler
|
|
|
|
**Location:** `internal/scheduler/quota_reset.go` (legacy name)
|
|
|
|
**Schedule:** `0 0 1 * *` (1st of month at midnight UTC) - configurable
|
|
|
|
**Purpose:** Reset monthly usage counters for billing cycles
|
|
|
|
**Process:**
|
|
1. Get all sites paginated (`GetAllSitesForUsageReset`)
|
|
2. For each site:
|
|
- Reset `SearchRequestsCount` to 0
|
|
- Reset `MonthlyPagesIndexed` to 0
|
|
- Set `LastResetAt` to current timestamp
|
|
- Update site in database
|
|
3. Log summary:
|
|
- Total sites processed
|
|
- Total sites reset
|
|
- Failed resets (if any)
|
|
|
|
**Configuration:**
|
|
```bash
|
|
# .env
|
|
SCHEDULER_QUOTA_RESET_ENABLED=true
|
|
SCHEDULER_QUOTA_RESET_SCHEDULE="0 0 1 * *"
|
|
```
|
|
|
|
**Use Case:** `ResetMonthlyUsageUseCase` (renamed from ResetMonthlyQuotasUseCase)
|
|
- Encapsulates reset logic for billing cycles
|
|
- Handles errors gracefully
|
|
- Logs progress
|
|
- No quota enforcement, only counter resets for accurate billing
|
|
|
|
**Production Considerations:**
|
|
- Runs in single backend instance (use distributed lock if multiple instances)
|
|
- Idempotent (safe to run multiple times)
|
|
- Paginated processing for large site counts
|
|
- Monitor logs for failures
|
|
- Critical for accurate billing - must run reliably
|
|
|
|
---
|
|
|
|
## Rate Limiting Architecture
|
|
|
|
### Overview
|
|
|
|
MaplePress implements a **Four-Tier Rate Limiting Architecture** to satisfy OWASP ASVS 4.2.2 requirements for anti-automation controls while supporting high-volume legitimate traffic for the core WordPress Plugin API business.
|
|
|
|
**CRITICAL REQUIREMENT**: Every new API endpoint MUST belong to one of the four rate limiters for OWASP compliance.
|
|
|
|
### OWASP Compliance
|
|
|
|
**OWASP ASVS 4.2.2**: "Verify that anti-automation controls are effective at mitigating breached credential testing, brute force, and account lockout attacks."
|
|
|
|
**CWE Coverage:**
|
|
- **CWE-307**: Improper Restriction of Excessive Authentication Attempts → Registration + Login rate limiters
|
|
- **CWE-770**: Allocation of Resources Without Limits or Throttling → Generic + Plugin API rate limiters
|
|
- **CWE-348**: Use of Less Trusted Source (IP validation) → clientip.Extractor with trusted proxy validation
|
|
- **CWE-532**: Insertion of Sensitive Information into Log File → Email/slug hashing for Redis keys
|
|
|
|
### The Four Rate Limiters
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Four-Tier Rate Limiting Architecture │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ 1. Registration Rate Limiter (CWE-307) │
|
|
│ - Scope: POST /api/v1/register │
|
|
│ - Strategy: IP-based │
|
|
│ - Default: 5 requests/hour per IP │
|
|
│ - Purpose: Prevent account farming, bot signups │
|
|
│ │
|
|
│ 2. Login Rate Limiter (CWE-307) │
|
|
│ - Scope: POST /api/v1/login │
|
|
│ - Strategy: Dual (IP + account with lockout) │
|
|
│ - Defaults: 10 attempts/15min (IP), 10 failed/30min lockout │
|
|
│ - Purpose: Prevent brute force, credential stuffing │
|
|
│ │
|
|
│ 3. Generic CRUD Rate Limiter (CWE-770) │
|
|
│ - Scope: Authenticated CRUD endpoints │
|
|
│ - Strategy: User-based (JWT user ID) │
|
|
│ - Default: 100 requests/hour per user │
|
|
│ - Purpose: Prevent resource exhaustion │
|
|
│ - Endpoints: tenants, users, sites, admin, /me, /hello │
|
|
│ │
|
|
│ 4. Plugin API Rate Limiter (CWE-770) │
|
|
│ - Scope: WordPress Plugin API endpoints │
|
|
│ - Strategy: Site-based (API key → site_id) │
|
|
│ - Default: 1000 requests/hour per site │
|
|
│ - Purpose: Core business protection with high throughput │
|
|
│ - Endpoints: /api/v1/plugin/* (7 endpoints) │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
#### 1. Registration Rate Limiter
|
|
|
|
**Configuration:**
|
|
```bash
|
|
RATELIMIT_REGISTRATION_ENABLED=true
|
|
RATELIMIT_REGISTRATION_MAX_REQUESTS=5
|
|
RATELIMIT_REGISTRATION_WINDOW=1h
|
|
```
|
|
|
|
**When to Use:**
|
|
- User registration endpoints
|
|
- Public account creation APIs
|
|
- IP-based protection needed
|
|
|
|
**Implementation:**
|
|
```go
|
|
// Apply to registration route
|
|
if s.config.RateLimit.RegistrationEnabled {
|
|
mux.HandleFunc("POST /api/v1/register",
|
|
s.rateLimitMiddlewares.Registration.Handler(
|
|
http.HandlerFunc(s.registerHandler.Handle),
|
|
).ServeHTTP)
|
|
}
|
|
```
|
|
|
|
#### 2. Login Rate Limiter
|
|
|
|
**Configuration:**
|
|
```bash
|
|
RATELIMIT_LOGIN_ENABLED=true
|
|
RATELIMIT_LOGIN_MAX_ATTEMPTS_PER_IP=10
|
|
RATELIMIT_LOGIN_IP_WINDOW=15m
|
|
RATELIMIT_LOGIN_MAX_FAILED_ATTEMPTS_PER_ACCOUNT=10
|
|
RATELIMIT_LOGIN_ACCOUNT_LOCKOUT_DURATION=30m
|
|
```
|
|
|
|
**When to Use:**
|
|
- User authentication endpoints
|
|
- Any endpoint accepting credentials
|
|
- Dual protection: IP-based + account lockout
|
|
|
|
**Implementation:**
|
|
```go
|
|
// Login handler handles rate limiting internally
|
|
// Uses specialized LoginRateLimiter with account lockout
|
|
func (h *LoginHandler) Handle(w http.ResponseWriter, r *http.Request) {
|
|
// Extract IP and email
|
|
// Check rate limits (IP + account)
|
|
// Handle login logic
|
|
}
|
|
```
|
|
|
|
#### 3. Generic CRUD Rate Limiter
|
|
|
|
**Configuration:**
|
|
```bash
|
|
RATELIMIT_GENERIC_ENABLED=true
|
|
RATELIMIT_GENERIC_MAX_REQUESTS=100
|
|
RATELIMIT_GENERIC_WINDOW=1h
|
|
```
|
|
|
|
**When to Use:**
|
|
- Authenticated CRUD endpoints (JWT)
|
|
- Tenant management routes
|
|
- User management routes
|
|
- Site management routes
|
|
- Admin routes
|
|
- Dashboard/profile routes
|
|
|
|
**Implementation:**
|
|
```go
|
|
// Helper method for JWT + Generic rate limiting
|
|
func (s *Server) applyAuthOnlyWithGenericRateLimit(handler http.HandlerFunc) http.HandlerFunc {
|
|
return func(w http.ResponseWriter, r *http.Request) {
|
|
// Chain: JWT validation → Auth check → Generic rate limit (user-based) → Handler
|
|
s.jwtMiddleware.Handler(
|
|
s.jwtMiddleware.RequireAuth(
|
|
s.rateLimitMiddlewares.Generic.HandlerWithUserKey(
|
|
http.HandlerFunc(handler),
|
|
),
|
|
),
|
|
).ServeHTTP(w, r)
|
|
}
|
|
}
|
|
|
|
// Apply to routes
|
|
if s.config.RateLimit.GenericEnabled {
|
|
mux.HandleFunc("GET /api/v1/me", s.applyAuthOnlyWithGenericRateLimit(s.getMeHandler.Handle))
|
|
mux.HandleFunc("POST /api/v1/tenants", s.applyAuthOnlyWithGenericRateLimit(s.createTenantHandler.Handle))
|
|
}
|
|
```
|
|
|
|
#### 4. Plugin API Rate Limiter
|
|
|
|
**Configuration:**
|
|
```bash
|
|
RATELIMIT_PLUGIN_API_ENABLED=true
|
|
RATELIMIT_PLUGIN_API_MAX_REQUESTS=1000
|
|
RATELIMIT_PLUGIN_API_WINDOW=1h
|
|
```
|
|
|
|
**When to Use:**
|
|
- WordPress Plugin API endpoints
|
|
- API key authenticated routes
|
|
- High-volume business-critical endpoints
|
|
- Site-based protection needed
|
|
|
|
**Implementation:**
|
|
```go
|
|
// Helper method for API Key + Plugin rate limiting
|
|
func (s *Server) applyAPIKeyAuthWithPluginRateLimit(handler http.HandlerFunc) http.HandlerFunc {
|
|
return func(w http.ResponseWriter, r *http.Request) {
|
|
// Chain: API key validation → Require API key → Plugin rate limit (site-based) → Handler
|
|
s.apikeyMiddleware.Handler(
|
|
s.apikeyMiddleware.RequireAPIKey(
|
|
s.rateLimitMiddlewares.PluginAPI.HandlerWithSiteKey(
|
|
http.HandlerFunc(handler),
|
|
),
|
|
),
|
|
).ServeHTTP(w, r)
|
|
}
|
|
}
|
|
|
|
// Apply to routes
|
|
if s.config.RateLimit.PluginAPIEnabled {
|
|
mux.HandleFunc("POST /api/v1/plugin/pages/sync", s.applyAPIKeyAuthWithPluginRateLimit(s.syncPagesHandler.Handle))
|
|
mux.HandleFunc("POST /api/v1/plugin/pages/search", s.applyAPIKeyAuthWithPluginRateLimit(s.searchPagesHandler.Handle))
|
|
}
|
|
```
|
|
|
|
### Adding Rate Limiting to New Endpoints
|
|
|
|
**Step 1: Identify the Endpoint Type**
|
|
|
|
Ask yourself:
|
|
1. Is this a public registration endpoint? → **Registration Rate Limiter**
|
|
2. Is this a login/authentication endpoint? → **Login Rate Limiter**
|
|
3. Is this a JWT-authenticated CRUD endpoint? → **Generic Rate Limiter**
|
|
4. Is this a WordPress Plugin API endpoint (API key auth)? → **Plugin API Rate Limiter**
|
|
|
|
**Step 2: Apply the Appropriate Rate Limiter**
|
|
|
|
**Example 1: New authenticated CRUD endpoint**
|
|
```go
|
|
// New endpoint: Update user profile
|
|
if s.config.RateLimit.GenericEnabled {
|
|
mux.HandleFunc("PUT /api/v1/users/{id}",
|
|
s.applyAuthAndTenantWithGenericRateLimit(s.updateUserHandler.Handle))
|
|
} else {
|
|
mux.HandleFunc("PUT /api/v1/users/{id}",
|
|
s.applyAuthAndTenant(s.updateUserHandler.Handle))
|
|
}
|
|
```
|
|
|
|
**Example 2: New WordPress Plugin API endpoint**
|
|
```go
|
|
// New endpoint: Get plugin statistics
|
|
if s.config.RateLimit.PluginAPIEnabled {
|
|
mux.HandleFunc("GET /api/v1/plugin/stats",
|
|
s.applyAPIKeyAuthWithPluginRateLimit(s.pluginStatsHandler.Handle))
|
|
} else {
|
|
mux.HandleFunc("GET /api/v1/plugin/stats",
|
|
s.applyAPIKeyAuth(s.pluginStatsHandler.Handle))
|
|
}
|
|
```
|
|
|
|
**Step 3: Test Rate Limiting**
|
|
|
|
```bash
|
|
# Test Generic Rate Limiter (100/hour limit)
|
|
TOKEN="your_jwt_token"
|
|
for i in {1..150}; do
|
|
curl http://localhost:8000/api/v1/me \
|
|
-H "Authorization: Bearer $TOKEN"
|
|
done
|
|
# Expected: First 100 succeed, rest return 429
|
|
|
|
# Test Plugin API Rate Limiter (1000/hour limit)
|
|
API_KEY="your_api_key"
|
|
for i in {1..1100}; do
|
|
curl http://localhost:8000/api/v1/plugin/status \
|
|
-H "Authorization: Bearer $API_KEY"
|
|
done
|
|
# Expected: First 1000 succeed, rest return 429
|
|
```
|
|
|
|
### Rate Limiting Strategies
|
|
|
|
#### IP-Based (Registration)
|
|
```go
|
|
// Redis key: ratelimit:registration:<ip_address>
|
|
// Extracts client IP using clientip.Extractor
|
|
// Sliding window algorithm
|
|
```
|
|
|
|
#### Dual (Login)
|
|
```go
|
|
// Redis keys:
|
|
// - login_rl:ip:<ip>
|
|
// - login_rl:account:<email_hash>:attempts
|
|
// - login_rl:account:<email_hash>:locked
|
|
// IP-based + account-based with lockout
|
|
// Specialized implementation in login handler
|
|
```
|
|
|
|
#### User-Based (Generic CRUD)
|
|
```go
|
|
// Redis key: ratelimit:generic:user:<user_id>
|
|
// Extracts user ID from JWT context (constants.SessionUserID)
|
|
// Fallback to IP if user ID not available
|
|
func (m *RateLimitMiddleware) HandlerWithUserKey(next http.Handler) http.Handler {
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
var key string
|
|
if userID, ok := r.Context().Value(constants.SessionUserID).(uint64); ok {
|
|
key = fmt.Sprintf("user:%d", userID)
|
|
} else {
|
|
// Fallback to IP-based rate limiting
|
|
key = fmt.Sprintf("ip:%s", m.ipExtractor.Extract(r))
|
|
}
|
|
// Check rate limit...
|
|
})
|
|
}
|
|
```
|
|
|
|
#### Site-Based (Plugin API)
|
|
```go
|
|
// Redis key: ratelimit:plugin:site:<site_id>
|
|
// Extracts site ID from API key context (constants.SiteID)
|
|
// Fallback to IP if site ID not available
|
|
func (m *RateLimitMiddleware) HandlerWithSiteKey(next http.Handler) http.Handler {
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
var key string
|
|
if siteID, ok := r.Context().Value(constants.SiteID).(string); ok && siteID != "" {
|
|
key = fmt.Sprintf("site:%s", siteID)
|
|
} else {
|
|
// Fallback to IP-based rate limiting
|
|
key = fmt.Sprintf("ip:%s", m.ipExtractor.Extract(r))
|
|
}
|
|
// Check rate limit...
|
|
})
|
|
}
|
|
```
|
|
|
|
### Configuration
|
|
|
|
**Environment Variables:**
|
|
|
|
```bash
|
|
# ============================================================================
|
|
# 1. Registration Rate Limiter (CWE-307: Account Creation Protection)
|
|
# ============================================================================
|
|
RATELIMIT_REGISTRATION_ENABLED=true
|
|
RATELIMIT_REGISTRATION_MAX_REQUESTS=5
|
|
RATELIMIT_REGISTRATION_WINDOW=1h
|
|
|
|
# ============================================================================
|
|
# 2. Login Rate Limiter (CWE-307: Brute Force Protection)
|
|
# ============================================================================
|
|
RATELIMIT_LOGIN_ENABLED=true
|
|
RATELIMIT_LOGIN_MAX_ATTEMPTS_PER_IP=10
|
|
RATELIMIT_LOGIN_IP_WINDOW=15m
|
|
RATELIMIT_LOGIN_MAX_FAILED_ATTEMPTS_PER_ACCOUNT=10
|
|
RATELIMIT_LOGIN_ACCOUNT_LOCKOUT_DURATION=30m
|
|
|
|
# ============================================================================
|
|
# 3. Generic CRUD Endpoints Rate Limiter (CWE-770: Resource Exhaustion Protection)
|
|
# ============================================================================
|
|
RATELIMIT_GENERIC_ENABLED=true
|
|
RATELIMIT_GENERIC_MAX_REQUESTS=100
|
|
RATELIMIT_GENERIC_WINDOW=1h
|
|
|
|
# ============================================================================
|
|
# 4. Plugin API Rate Limiter (CWE-770: DoS Prevention for Core Business)
|
|
# ============================================================================
|
|
RATELIMIT_PLUGIN_API_ENABLED=true
|
|
RATELIMIT_PLUGIN_API_MAX_REQUESTS=1000
|
|
RATELIMIT_PLUGIN_API_WINDOW=1h
|
|
```
|
|
|
|
**Recommended Production Values:**
|
|
|
|
**Registration** (most strict):
|
|
- Small sites: 5 requests/hour per IP ✅ (default)
|
|
- Medium sites: 10 requests/hour per IP
|
|
- Large sites: 20 requests/hour per IP
|
|
|
|
**Login** (moderate):
|
|
- Default: 10 attempts/15min per IP ✅
|
|
- Stricter: 5 attempts/10min per IP
|
|
- More lenient: 15 attempts/30min per IP
|
|
|
|
**Generic CRUD** (lenient):
|
|
- Default: 100 requests/hour per user ✅
|
|
- Heavy usage: 200 requests/hour per user
|
|
- Very heavy: 500 requests/hour per user
|
|
|
|
**Plugin API** (very lenient - core business):
|
|
- Default: 1000 requests/hour per site ✅
|
|
- Enterprise tier: 5000 requests/hour per site
|
|
- Premium tier: 10000 requests/hour per site
|
|
|
|
### Endpoint Coverage
|
|
|
|
**Total API Endpoints**: 25
|
|
**Endpoints with Rate Limiting**: 23 (92%)
|
|
**OWASP-Critical Endpoints Protected**: 23/23 (100%)
|
|
|
|
#### Registration Rate Limiter (1 endpoint)
|
|
- `POST /api/v1/register` - IP-based, 5/hour
|
|
|
|
#### Login Rate Limiter (1 endpoint)
|
|
- `POST /api/v1/login` - Dual (IP + account lockout), 10/15min per IP
|
|
|
|
#### Generic CRUD Rate Limiter (15 endpoints)
|
|
User-based, 100/hour per user:
|
|
- `POST /api/v1/hello`, `GET /api/v1/me`
|
|
- `POST /api/v1/tenants`, `GET /api/v1/tenants/{id}`, `GET /api/v1/tenants/slug/{slug}`
|
|
- `POST /api/v1/users`, `GET /api/v1/users/{id}`
|
|
- `POST /api/v1/sites`, `GET /api/v1/sites`, `GET /api/v1/sites/{id}`, `DELETE /api/v1/sites/{id}`, `POST /api/v1/sites/{id}/rotate-api-key`
|
|
- `POST /api/v1/admin/unlock-account`, `GET /api/v1/admin/account-status`
|
|
|
|
#### Plugin API Rate Limiter (7 endpoints)
|
|
Site-based, 1000/hour per site:
|
|
- `GET /api/v1/plugin/status`
|
|
- `POST /api/v1/plugin/pages/sync`
|
|
- `POST /api/v1/plugin/pages/search`
|
|
- `DELETE /api/v1/plugin/pages`
|
|
- `DELETE /api/v1/plugin/pages/all`
|
|
- `GET /api/v1/plugin/pages/status`
|
|
- `GET /api/v1/plugin/pages/{page_id}`
|
|
|
|
#### No Rate Limiting (2 endpoints)
|
|
- `GET /health` - Health check endpoint (no rate limit needed)
|
|
- `POST /api/v1/refresh` - Token refresh (no rate limit needed, short-lived)
|
|
|
|
### Fail-Open Design
|
|
|
|
All rate limiters implement **fail-open design**:
|
|
- If Redis is down, requests are **allowed**
|
|
- Error is logged, but request proceeds
|
|
- Prioritizes availability over strict security
|
|
- Appropriate for business-critical endpoints
|
|
|
|
```go
|
|
// Check rate limit
|
|
allowed, err := m.rateLimiter.Allow(r.Context(), key)
|
|
if err != nil {
|
|
// Log error but allow request (fail-open)
|
|
m.logger.Error("rate limiter error",
|
|
zap.String("key", key),
|
|
zap.Error(err))
|
|
}
|
|
|
|
if !allowed {
|
|
// Rate limit exceeded
|
|
w.Header().Set("Retry-After", "3600") // 1 hour
|
|
httperror.TooManyRequests(w, "Rate limit exceeded. Please try again later.")
|
|
return
|
|
}
|
|
```
|
|
|
|
### Monitoring and Troubleshooting
|
|
|
|
**Check Rate Limiter Initialization:**
|
|
```bash
|
|
# View logs for rate limiter initialization
|
|
docker logs mapleopentech_backend | grep "rate"
|
|
|
|
# Expected output:
|
|
# Registration rate limiter: enabled=true, max_requests=5, window=1h0m0s
|
|
# Login rate limiter: enabled=true, max_attempts_ip=10, ip_window=15m0s
|
|
# Generic rate limiter: enabled=true, max_requests=100, window=1h0m0s
|
|
# Plugin API rate limiter: enabled=true, max_requests=1000, window=1h0m0s
|
|
```
|
|
|
|
**Check Redis Keys:**
|
|
```bash
|
|
# Connect to Redis
|
|
docker exec -it mapleopentech_redis redis-cli
|
|
|
|
# List rate limit keys
|
|
KEYS ratelimit:*
|
|
KEYS login_rl:*
|
|
|
|
# Get rate limit value for specific key
|
|
GET ratelimit:registration:<ip_address>
|
|
GET ratelimit:generic:user:<user_id>
|
|
GET ratelimit:plugin:site:<site_id>
|
|
```
|
|
|
|
**Disable Rate Limiter Temporarily:**
|
|
```bash
|
|
# Disable specific rate limiter in .env
|
|
RATELIMIT_GENERIC_ENABLED=false
|
|
RATELIMIT_PLUGIN_API_ENABLED=false
|
|
|
|
# Restart backend
|
|
task end && task dev
|
|
```
|
|
|
|
### Common Pitfalls
|
|
|
|
#### ❌ Forgetting to apply rate limiting to new endpoints
|
|
|
|
**Wrong:**
|
|
```go
|
|
// New endpoint without rate limiting (OWASP violation!)
|
|
mux.HandleFunc("POST /api/v1/posts", s.createPostHandler.Handle)
|
|
```
|
|
|
|
**Correct:**
|
|
```go
|
|
// New endpoint with appropriate rate limiting
|
|
if s.config.RateLimit.GenericEnabled {
|
|
mux.HandleFunc("POST /api/v1/posts",
|
|
s.applyAuthAndTenantWithGenericRateLimit(s.createPostHandler.Handle))
|
|
} else {
|
|
mux.HandleFunc("POST /api/v1/posts",
|
|
s.applyAuthAndTenant(s.createPostHandler.Handle))
|
|
}
|
|
```
|
|
|
|
#### ❌ Using wrong rate limiter for endpoint type
|
|
|
|
**Wrong:**
|
|
```go
|
|
// Using Generic rate limiter for WordPress Plugin API
|
|
mux.HandleFunc("POST /api/v1/plugin/pages/sync",
|
|
s.applyAuthOnlyWithGenericRateLimit(s.syncPagesHandler.Handle))
|
|
```
|
|
|
|
**Correct:**
|
|
```go
|
|
// Using Plugin API rate limiter for WordPress Plugin API
|
|
mux.HandleFunc("POST /api/v1/plugin/pages/sync",
|
|
s.applyAPIKeyAuthWithPluginRateLimit(s.syncPagesHandler.Handle))
|
|
```
|
|
|
|
#### ❌ Missing configuration check
|
|
|
|
**Wrong:**
|
|
```go
|
|
// Always applying rate limiting (doesn't respect config)
|
|
mux.HandleFunc("GET /api/v1/me",
|
|
s.applyAuthOnlyWithGenericRateLimit(s.getMeHandler.Handle))
|
|
```
|
|
|
|
**Correct:**
|
|
```go
|
|
// Respecting configuration flag
|
|
if s.config.RateLimit.GenericEnabled {
|
|
mux.HandleFunc("GET /api/v1/me",
|
|
s.applyAuthOnlyWithGenericRateLimit(s.getMeHandler.Handle))
|
|
} else {
|
|
mux.HandleFunc("GET /api/v1/me",
|
|
s.applyAuthOnly(s.getMeHandler.Handle))
|
|
}
|
|
```
|
|
|
|
### Summary
|
|
|
|
**Key Takeaways:**
|
|
1. ✅ Every new API endpoint MUST belong to one of the four rate limiters
|
|
2. ✅ Choose the appropriate rate limiter based on endpoint type
|
|
3. ✅ Always wrap routes with configuration checks
|
|
4. ✅ Use helper methods for consistent middleware chaining
|
|
5. ✅ Test rate limiting after adding new endpoints
|
|
6. ✅ Monitor Redis for rate limit key usage
|
|
7. ✅ OWASP ASVS 4.2.2 compliance is mandatory
|
|
|
|
**Files to Modify When Adding New Endpoints:**
|
|
1. `internal/interface/http/server.go` - Add route with rate limiting
|
|
2. Test your endpoint with rate limit testing script
|
|
|
|
---
|
|
|
|
## Multi-Tenancy Implementation
|
|
|
|
### Overview
|
|
|
|
MaplePress uses **shared tables with tenant isolation via partition keys**. This means:
|
|
- All tenants share the same Cassandra cluster and tables
|
|
- Each row has a `tenant_id` field in its partition key
|
|
- Cassandra physically separates data by partition key (tenant A's data on different nodes than tenant B)
|
|
- ALL repository methods require `tenantID` parameter
|
|
|
|
### Tenant Extraction
|
|
|
|
Tenant ID is extracted from HTTP headers by middleware:
|
|
|
|
**File**: `internal/interface/http/middleware/tenant.go`
|
|
|
|
```go
|
|
// For development: get from X-Tenant-ID header
|
|
// TODO: In production, extract from JWT token
|
|
tenantID := r.Header.Get("X-Tenant-ID")
|
|
|
|
// Store in context
|
|
ctx := context.WithValue(r.Context(), TenantIDKey, tenantID)
|
|
```
|
|
|
|
**Production TODO**: Replace header extraction with JWT token validation.
|
|
|
|
### Repository Pattern
|
|
|
|
**ALL repository methods require tenantID**:
|
|
|
|
```go
|
|
type Repository interface {
|
|
Create(ctx context.Context, tenantID string, user *User) error
|
|
GetByID(ctx context.Context, tenantID string, id string) (*User, error)
|
|
GetByEmail(ctx context.Context, tenantID string, email string) (*User, error)
|
|
Update(ctx context.Context, tenantID string, user *User) error
|
|
Delete(ctx context.Context, tenantID string, id string) error
|
|
}
|
|
```
|
|
|
|
### Table Models with tenant_id
|
|
|
|
**File**: `internal/repository/user/models/user_by_id.go`
|
|
|
|
```go
|
|
type UserByID struct {
|
|
TenantID string `db:"tenant_id"` // Multi-tenant isolation
|
|
ID string `db:"id"`
|
|
Email string `db:"email"`
|
|
Name string `db:"name"`
|
|
CreatedAt time.Time `db:"created_at"`
|
|
UpdatedAt time.Time `db:"updated_at"`
|
|
}
|
|
|
|
// Conversion functions require tenantID
|
|
func FromUser(tenantID string, u *user.User) *UserByID {
|
|
return &UserByID{
|
|
TenantID: tenantID, // CRITICAL: Set tenant ID
|
|
ID: u.ID,
|
|
Email: u.Email,
|
|
// ...
|
|
}
|
|
}
|
|
```
|
|
|
|
### Queries with tenant_id
|
|
|
|
```go
|
|
// CORRECT: Query with tenant_id
|
|
query := `SELECT tenant_id, id, email, name, created_at, updated_at
|
|
FROM users_by_id
|
|
WHERE tenant_id = ? AND id = ?`
|
|
|
|
// WRONG: Query without tenant_id (would fail - requires table scan)
|
|
query := `SELECT id, email FROM users_by_id WHERE id = ?`
|
|
```
|
|
|
|
---
|
|
|
|
## Working with Cassandra
|
|
|
|
### Query-First Data Modeling
|
|
|
|
Cassandra requires designing tables for specific query patterns, not normalizing data.
|
|
|
|
**Rule**: One query pattern = One table
|
|
|
|
**Example**:
|
|
```
|
|
Query 1: Get user by ID → users_by_id table
|
|
Query 2: Get user by email → users_by_email table
|
|
Query 3: List users by date → users_by_date table
|
|
```
|
|
|
|
### Primary Keys
|
|
|
|
**Partition Key**: Determines which node stores the data
|
|
**Clustering Key**: Sorts data within a partition
|
|
|
|
```cql
|
|
-- Partition key: (tenant_id, id)
|
|
-- No clustering key
|
|
PRIMARY KEY ((tenant_id, id))
|
|
|
|
-- Partition key: (tenant_id, created_date)
|
|
-- Clustering key: id
|
|
PRIMARY KEY ((tenant_id, created_date), id)
|
|
```
|
|
|
|
### Batched Writes
|
|
|
|
When writing to multiple tables (denormalization), use batched writes for consistency:
|
|
|
|
**File**: `internal/repository/user/create.go`
|
|
|
|
```go
|
|
func (r *repository) Create(ctx context.Context, tenantID string, u *domainuser.User) error {
|
|
// Convert to table models
|
|
userByID := models.FromUser(tenantID, u)
|
|
userByEmail := models.FromUserByEmail(tenantID, u)
|
|
userByDate := models.FromUserByDate(tenantID, u)
|
|
|
|
// Create batch (atomic write to all 3 tables)
|
|
batch := r.session.NewBatch(gocql.LoggedBatch)
|
|
|
|
// Add all writes to batch
|
|
batch.Query(`INSERT INTO users_by_id (...) VALUES (...)`, ...)
|
|
batch.Query(`INSERT INTO users_by_email (...) VALUES (...)`, ...)
|
|
batch.Query(`INSERT INTO users_by_date (...) VALUES (...)`, ...)
|
|
|
|
// Execute atomically
|
|
return r.session.ExecuteBatch(batch)
|
|
}
|
|
```
|
|
|
|
**Rule**: ALWAYS use batched writes for create/update/delete to maintain consistency across denormalized tables.
|
|
|
|
### Consistency Levels
|
|
|
|
MaplePress uses `QUORUM` consistency by default (defined in config):
|
|
|
|
```
|
|
QUORUM = (Replication Factor / 2) + 1
|
|
With RF=3: QUORUM = 2 nodes must acknowledge
|
|
```
|
|
|
|
This balances consistency and availability.
|
|
|
|
---
|
|
|
|
## Adding New Features
|
|
|
|
### Quick Reference Checklist
|
|
|
|
When adding a new entity (e.g., "Post"):
|
|
|
|
- [ ] 1. Define domain entity: `internal/domain/post/entity.go`
|
|
- [ ] 2. Define repository interface: `internal/domain/post/repository.go`
|
|
- [ ] 3. Design Cassandra tables (one per query pattern)
|
|
- [ ] 4. Create table models: `internal/repository/post/models/`
|
|
- [ ] 5. Implement repository: `internal/repository/post/`
|
|
- [ ] 6. Create schema file: `internal/repository/post/schema.cql`
|
|
- [ ] 7. Implement use cases: `internal/usecase/post/`
|
|
- [ ] 8. Create service: `internal/service/post_service.go`
|
|
- [ ] 9. Create DTOs: `internal/interface/http/dto/post/`
|
|
- [ ] 10. Implement handlers: `internal/interface/http/handler/post/`
|
|
- [ ] 11. Wire everything: `app/wire.go`
|
|
- [ ] 12. Add routes: `internal/interface/http/server.go`
|
|
|
|
### Step-by-Step: Adding "Post" Entity
|
|
|
|
#### Step 1: Domain Layer
|
|
|
|
**File**: `internal/domain/post/entity.go`
|
|
|
|
```go
|
|
package post
|
|
|
|
import (
|
|
"errors"
|
|
"time"
|
|
)
|
|
|
|
var (
|
|
ErrTitleRequired = errors.New("title is required")
|
|
ErrContentRequired = errors.New("content is required")
|
|
)
|
|
|
|
type Post struct {
|
|
ID string
|
|
TenantID string // Not in domain, but needed for multi-tenancy
|
|
AuthorID string
|
|
Title string
|
|
Content string
|
|
CreatedAt time.Time
|
|
UpdatedAt time.Time
|
|
}
|
|
|
|
func (p *Post) Validate() error {
|
|
if p.Title == "" {
|
|
return ErrTitleRequired
|
|
}
|
|
if p.Content == "" {
|
|
return ErrContentRequired
|
|
}
|
|
return nil
|
|
}
|
|
```
|
|
|
|
**File**: `internal/domain/post/repository.go`
|
|
|
|
```go
|
|
package post
|
|
|
|
import "context"
|
|
|
|
// Repository defines data access for posts
|
|
// All methods require tenantID for multi-tenant isolation
|
|
type Repository interface {
|
|
Create(ctx context.Context, tenantID string, post *Post) error
|
|
GetByID(ctx context.Context, tenantID string, id string) (*Post, error)
|
|
Update(ctx context.Context, tenantID string, post *Post) error
|
|
Delete(ctx context.Context, tenantID string, id string) error
|
|
ListByAuthor(ctx context.Context, tenantID string, authorID string) ([]*Post, error)
|
|
}
|
|
```
|
|
|
|
#### Step 2: Design Cassandra Tables
|
|
|
|
Identify query patterns:
|
|
1. Get post by ID
|
|
2. List posts by author
|
|
|
|
**File**: `internal/repository/post/schema.cql`
|
|
|
|
```cql
|
|
-- posts_by_id: Get post by ID
|
|
CREATE TABLE IF NOT EXISTS posts_by_id (
|
|
tenant_id UUID,
|
|
id UUID,
|
|
author_id UUID,
|
|
title TEXT,
|
|
content TEXT,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP,
|
|
PRIMARY KEY ((tenant_id, id))
|
|
);
|
|
|
|
-- posts_by_author: List posts by author
|
|
CREATE TABLE IF NOT EXISTS posts_by_author (
|
|
tenant_id UUID,
|
|
author_id UUID,
|
|
id UUID,
|
|
title TEXT,
|
|
content TEXT,
|
|
created_at TIMESTAMP,
|
|
updated_at TIMESTAMP,
|
|
PRIMARY KEY ((tenant_id, author_id), created_at, id)
|
|
) WITH CLUSTERING ORDER BY (created_at DESC, id ASC);
|
|
```
|
|
|
|
#### Step 3: Create Table Models
|
|
|
|
**File**: `internal/repository/post/models/post_by_id.go`
|
|
|
|
```go
|
|
package models
|
|
|
|
import (
|
|
"time"
|
|
"codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/domain/post"
|
|
)
|
|
|
|
// PostByID represents the posts_by_id table
|
|
// Query pattern: Get post by ID
|
|
type PostByID struct {
|
|
TenantID string `db:"tenant_id"`
|
|
ID string `db:"id"`
|
|
AuthorID string `db:"author_id"`
|
|
Title string `db:"title"`
|
|
Content string `db:"content"`
|
|
CreatedAt time.Time `db:"created_at"`
|
|
UpdatedAt time.Time `db:"updated_at"`
|
|
}
|
|
|
|
func (p *PostByID) ToPost() *post.Post {
|
|
return &post.Post{
|
|
ID: p.ID,
|
|
AuthorID: p.AuthorID,
|
|
Title: p.Title,
|
|
Content: p.Content,
|
|
CreatedAt: p.CreatedAt,
|
|
UpdatedAt: p.UpdatedAt,
|
|
}
|
|
}
|
|
|
|
func FromPost(tenantID string, p *post.Post) *PostByID {
|
|
return &PostByID{
|
|
TenantID: tenantID,
|
|
ID: p.ID,
|
|
AuthorID: p.AuthorID,
|
|
Title: p.Title,
|
|
Content: p.Content,
|
|
CreatedAt: p.CreatedAt,
|
|
UpdatedAt: p.UpdatedAt,
|
|
}
|
|
}
|
|
```
|
|
|
|
**File**: `internal/repository/post/models/post_by_author.go`
|
|
|
|
```go
|
|
package models
|
|
|
|
import (
|
|
"time"
|
|
"codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/domain/post"
|
|
)
|
|
|
|
// PostByAuthor represents the posts_by_author table
|
|
// Query pattern: List posts by author
|
|
type PostByAuthor struct {
|
|
TenantID string `db:"tenant_id"`
|
|
AuthorID string `db:"author_id"`
|
|
ID string `db:"id"`
|
|
Title string `db:"title"`
|
|
Content string `db:"content"`
|
|
CreatedAt time.Time `db:"created_at"`
|
|
UpdatedAt time.Time `db:"updated_at"`
|
|
}
|
|
|
|
func (p *PostByAuthor) ToPost() *post.Post {
|
|
return &post.Post{
|
|
ID: p.ID,
|
|
AuthorID: p.AuthorID,
|
|
Title: p.Title,
|
|
Content: p.Content,
|
|
CreatedAt: p.CreatedAt,
|
|
UpdatedAt: p.UpdatedAt,
|
|
}
|
|
}
|
|
|
|
func FromPostByAuthor(tenantID string, p *post.Post) *PostByAuthor {
|
|
return &PostByAuthor{
|
|
TenantID: tenantID,
|
|
AuthorID: p.AuthorID,
|
|
ID: p.ID,
|
|
Title: p.Title,
|
|
Content: p.Content,
|
|
CreatedAt: p.CreatedAt,
|
|
UpdatedAt: p.UpdatedAt,
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Step 4: Implement Repository
|
|
|
|
**File**: `internal/repository/post/impl.go`
|
|
|
|
```go
|
|
package post
|
|
|
|
import (
|
|
"github.com/gocql/gocql"
|
|
"go.uber.org/zap"
|
|
|
|
domainpost "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/domain/post"
|
|
)
|
|
|
|
type repository struct {
|
|
session *gocql.Session
|
|
logger *zap.Logger
|
|
}
|
|
|
|
func ProvideRepository(session *gocql.Session, logger *zap.Logger) domainpost.Repository {
|
|
return &repository{
|
|
session: session,
|
|
logger: logger,
|
|
}
|
|
}
|
|
```
|
|
|
|
**File**: `internal/repository/post/create.go`
|
|
|
|
```go
|
|
package post
|
|
|
|
import (
|
|
"context"
|
|
"github.com/gocql/gocql"
|
|
|
|
domainpost "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/domain/post"
|
|
"codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/repository/post/models"
|
|
)
|
|
|
|
func (r *repository) Create(ctx context.Context, tenantID string, p *domainpost.Post) error {
|
|
// Convert to table models
|
|
postByID := models.FromPost(tenantID, p)
|
|
postByAuthor := models.FromPostByAuthor(tenantID, p)
|
|
|
|
// Batched write for consistency
|
|
batch := r.session.NewBatch(gocql.LoggedBatch)
|
|
|
|
batch.Query(`INSERT INTO posts_by_id (tenant_id, id, author_id, title, content, created_at, updated_at)
|
|
VALUES (?, ?, ?, ?, ?, ?, ?)`,
|
|
postByID.TenantID, postByID.ID, postByID.AuthorID, postByID.Title,
|
|
postByID.Content, postByID.CreatedAt, postByID.UpdatedAt)
|
|
|
|
batch.Query(`INSERT INTO posts_by_author (tenant_id, author_id, id, title, content, created_at, updated_at)
|
|
VALUES (?, ?, ?, ?, ?, ?, ?)`,
|
|
postByAuthor.TenantID, postByAuthor.AuthorID, postByAuthor.ID, postByAuthor.Title,
|
|
postByAuthor.Content, postByAuthor.CreatedAt, postByAuthor.UpdatedAt)
|
|
|
|
return r.session.ExecuteBatch(batch)
|
|
}
|
|
```
|
|
|
|
#### Step 5: Implement Use Cases
|
|
|
|
**File**: `internal/usecase/post/create.go`
|
|
|
|
```go
|
|
package post
|
|
|
|
import (
|
|
"context"
|
|
"time"
|
|
"github.com/google/uuid"
|
|
|
|
domainpost "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/domain/post"
|
|
)
|
|
|
|
type CreatePostInput struct {
|
|
AuthorID string
|
|
Title string
|
|
Content string
|
|
}
|
|
|
|
type CreatePostOutput struct {
|
|
ID string
|
|
Title string
|
|
CreatedAt time.Time
|
|
}
|
|
|
|
type CreatePostUseCase struct {
|
|
repo domainpost.Repository
|
|
}
|
|
|
|
func ProvideCreatePostUseCase(repo domainpost.Repository) *CreatePostUseCase {
|
|
return &CreatePostUseCase{repo: repo}
|
|
}
|
|
|
|
func (uc *CreatePostUseCase) Execute(ctx context.Context, tenantID string, input *CreatePostInput) (*CreatePostOutput, error) {
|
|
now := time.Now()
|
|
|
|
post := &domainpost.Post{
|
|
ID: uuid.New().String(),
|
|
AuthorID: input.AuthorID,
|
|
Title: input.Title,
|
|
Content: input.Content,
|
|
CreatedAt: now,
|
|
UpdatedAt: now,
|
|
}
|
|
|
|
if err := post.Validate(); err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
if err := uc.repo.Create(ctx, tenantID, post); err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
return &CreatePostOutput{
|
|
ID: post.ID,
|
|
Title: post.Title,
|
|
CreatedAt: post.CreatedAt,
|
|
}, nil
|
|
}
|
|
```
|
|
|
|
#### Step 6: Create Service
|
|
|
|
**File**: `internal/service/post_service.go`
|
|
|
|
```go
|
|
package service
|
|
|
|
import (
|
|
"context"
|
|
"go.uber.org/zap"
|
|
|
|
postupc "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/usecase/post"
|
|
)
|
|
|
|
type PostService struct {
|
|
createUC *postupc.CreatePostUseCase
|
|
logger *zap.Logger
|
|
}
|
|
|
|
func ProvidePostService(
|
|
createUC *postupc.CreatePostUseCase,
|
|
logger *zap.Logger,
|
|
) *PostService {
|
|
return &PostService{
|
|
createUC: createUC,
|
|
logger: logger,
|
|
}
|
|
}
|
|
|
|
func (s *PostService) CreatePost(ctx context.Context, tenantID string, input *postupc.CreatePostInput) (*postupc.CreatePostOutput, error) {
|
|
return s.createUC.Execute(ctx, tenantID, input)
|
|
}
|
|
```
|
|
|
|
#### Step 7: Create DTOs
|
|
|
|
**File**: `internal/interface/http/dto/post/create_dto.go`
|
|
|
|
```go
|
|
package post
|
|
|
|
import "time"
|
|
|
|
type CreateRequest struct {
|
|
AuthorID string `json:"author_id"`
|
|
Title string `json:"title"`
|
|
Content string `json:"content"`
|
|
}
|
|
|
|
type CreateResponse struct {
|
|
ID string `json:"id"`
|
|
Title string `json:"title"`
|
|
CreatedAt time.Time `json:"created_at"`
|
|
}
|
|
```
|
|
|
|
#### Step 8: Create Handler
|
|
|
|
**File**: `internal/interface/http/handler/post/create_handler.go`
|
|
|
|
```go
|
|
package post
|
|
|
|
import (
|
|
"encoding/json"
|
|
"net/http"
|
|
"go.uber.org/zap"
|
|
|
|
postdto "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/interface/http/dto/post"
|
|
"codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/interface/http/middleware"
|
|
"codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/service"
|
|
postupc "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/usecase/post"
|
|
"codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/pkg/httperror"
|
|
)
|
|
|
|
type CreateHandler struct {
|
|
service *service.PostService
|
|
logger *zap.Logger
|
|
}
|
|
|
|
func ProvideCreateHandler(service *service.PostService, logger *zap.Logger) *CreateHandler {
|
|
return &CreateHandler{
|
|
service: service,
|
|
logger: logger,
|
|
}
|
|
}
|
|
|
|
func (h *CreateHandler) Handle(w http.ResponseWriter, r *http.Request) {
|
|
tenantID, err := middleware.GetTenantID(r.Context())
|
|
if err != nil {
|
|
httperror.Unauthorized(w, "missing tenant")
|
|
return
|
|
}
|
|
|
|
var req postdto.CreateRequest
|
|
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
|
httperror.BadRequest(w, "invalid request body")
|
|
return
|
|
}
|
|
|
|
input := &postupc.CreatePostInput{
|
|
AuthorID: req.AuthorID,
|
|
Title: req.Title,
|
|
Content: req.Content,
|
|
}
|
|
|
|
output, err := h.service.CreatePost(r.Context(), tenantID, input)
|
|
if err != nil {
|
|
h.logger.Error("failed to create post", zap.Error(err))
|
|
httperror.InternalServerError(w, "failed to create post")
|
|
return
|
|
}
|
|
|
|
response := postdto.CreateResponse{
|
|
ID: output.ID,
|
|
Title: output.Title,
|
|
CreatedAt: output.CreatedAt,
|
|
}
|
|
|
|
w.Header().Set("Content-Type", "application/json")
|
|
w.WriteHeader(http.StatusCreated)
|
|
json.NewEncoder(w).Encode(response)
|
|
}
|
|
```
|
|
|
|
#### Step 9: Wire Dependencies
|
|
|
|
**File**: `app/wire.go`
|
|
|
|
```go
|
|
// Add to imports
|
|
postrepo "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/repository/post"
|
|
postupc "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/usecase/post"
|
|
posthandler "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/internal/interface/http/handler/post"
|
|
|
|
// Add to wire.Build()
|
|
wire.Build(
|
|
// ... existing providers ...
|
|
|
|
// Post repository
|
|
postrepo.ProvideRepository,
|
|
|
|
// Post use cases
|
|
postupc.ProvideCreatePostUseCase,
|
|
|
|
// Post service
|
|
service.ProvidePostService,
|
|
|
|
// Post handlers
|
|
posthandler.ProvideCreateHandler,
|
|
|
|
// ... rest ...
|
|
)
|
|
```
|
|
|
|
**Update**: `internal/service/post_service.go` provider function
|
|
|
|
**Update**: `internal/interface/http/server.go` struct to include post handlers
|
|
|
|
#### Step 10: Add Routes
|
|
|
|
**File**: `internal/interface/http/server.go`
|
|
|
|
```go
|
|
// Add to Server struct
|
|
type Server struct {
|
|
// ... existing ...
|
|
createPostHandler *posthandler.CreateHandler
|
|
}
|
|
|
|
// Update ProvideServer
|
|
func ProvideServer(
|
|
cfg *config.Config,
|
|
logger *zap.Logger,
|
|
healthHandler *healthcheck.HealthCheckHandler,
|
|
createUserHandler *userhandler.CreateHandler,
|
|
getUserHandler *userhandler.GetHandler,
|
|
createPostHandler *posthandler.CreateHandler, // NEW
|
|
) *Server {
|
|
return &Server{
|
|
// ... existing ...
|
|
createPostHandler: createPostHandler,
|
|
}
|
|
}
|
|
|
|
// Add to registerRoutes
|
|
func (s *Server) registerRoutes(mux *http.ServeMux) {
|
|
// ... existing routes ...
|
|
mux.HandleFunc("POST /api/v1/posts", s.createPostHandler.Handle)
|
|
}
|
|
```
|
|
|
|
#### Step 11: Test Your Changes
|
|
|
|
```bash
|
|
# The dev server auto-rebuilds via CompileDaemon
|
|
# Just save your files and check the logs
|
|
|
|
# Or restart the dev server to see changes
|
|
task dev
|
|
```
|
|
|
|
---
|
|
|
|
## Code Patterns & Conventions
|
|
|
|
### File Organization
|
|
|
|
**One operation per file**:
|
|
```
|
|
internal/repository/user/
|
|
├── impl.go # Repository struct
|
|
├── create.go # Create operation
|
|
├── get.go # Get operations (GetByID, GetByEmail)
|
|
├── update.go # Update operation
|
|
├── delete.go # Delete operation
|
|
└── list.go # List operations
|
|
```
|
|
|
|
### Naming Conventions
|
|
|
|
**Repository Methods**: Verb + preposition
|
|
```go
|
|
Create()
|
|
GetByID()
|
|
GetByEmail()
|
|
Update()
|
|
Delete()
|
|
ListByDate()
|
|
ListByAuthor()
|
|
```
|
|
|
|
**Use Case Files**: `[operation].go`
|
|
```
|
|
internal/usecase/user/
|
|
├── create.go
|
|
├── get.go
|
|
├── update.go
|
|
└── delete.go
|
|
```
|
|
|
|
**DTOs**: `[operation]_dto.go`
|
|
```
|
|
internal/interface/http/dto/user/
|
|
├── create_dto.go
|
|
├── get_dto.go
|
|
└── update_dto.go
|
|
```
|
|
|
|
### Import Aliases
|
|
|
|
Use aliases to avoid conflicts:
|
|
```go
|
|
import (
|
|
userdto "path/to/dto/user"
|
|
userusecase "path/to/usecase/user"
|
|
userrepo "path/to/repository/user"
|
|
domainuser "path/to/domain/user"
|
|
)
|
|
```
|
|
|
|
### Error Handling
|
|
|
|
#### RFC 9457 (Problem Details for HTTP APIs) ✅
|
|
|
|
MaplePress implements **RFC 9457** (previously RFC 7807) for standardized HTTP error responses. This provides machine-readable, structured error responses that clients can easily parse and display.
|
|
|
|
**Standard**: [RFC 9457 - Problem Details for HTTP APIs](https://datatracker.ietf.org/doc/html/rfc9457)
|
|
|
|
**Implementation Location**: `pkg/httperror/error.go`
|
|
|
|
**Response Structure**:
|
|
```go
|
|
type ProblemDetail struct {
|
|
Type string `json:"type"` // URI reference identifying the problem type
|
|
Title string `json:"title"` // Short, human-readable summary
|
|
Status int `json:"status"` // HTTP status code
|
|
Detail string `json:"detail,omitempty"` // Human-readable explanation
|
|
Instance string `json:"instance,omitempty"` // URI reference to specific occurrence
|
|
Errors map[string][]string `json:"errors,omitempty"` // Validation errors (extension field)
|
|
}
|
|
```
|
|
|
|
**Content-Type**: All RFC 9457 responses use `application/problem+json`
|
|
|
|
**Usage - Validation Errors**:
|
|
```go
|
|
// For validation errors with field-specific messages
|
|
validationErrors := map[string][]string{
|
|
"email": {"Invalid email format"},
|
|
"password": {"Field is required", "Password must be at least 8 characters"},
|
|
"name": {"Field is required"},
|
|
}
|
|
|
|
httperror.ValidationError(w, validationErrors, "One or more validation errors occurred")
|
|
```
|
|
|
|
**Example Response**:
|
|
```json
|
|
{
|
|
"type": "about:blank",
|
|
"title": "Validation Error",
|
|
"status": 400,
|
|
"detail": "One or more validation errors occurred",
|
|
"errors": {
|
|
"email": ["Invalid email format"],
|
|
"password": ["Field is required", "Password must be at least 8 characters"],
|
|
"name": ["Field is required"]
|
|
}
|
|
}
|
|
```
|
|
|
|
**Usage - Simple Errors**:
|
|
```go
|
|
// For simple errors without field-specific details
|
|
httperror.ProblemBadRequest(w, "Invalid request body")
|
|
httperror.ProblemUnauthorized(w, "Authentication required")
|
|
httperror.ProblemForbidden(w, "Access denied")
|
|
httperror.ProblemNotFound(w, "User not found")
|
|
httperror.ProblemConflict(w, "Email already exists")
|
|
httperror.ProblemTooManyRequests(w, "Rate limit exceeded")
|
|
httperror.ProblemInternalServerError(w, "Failed to create user")
|
|
```
|
|
|
|
**Available Helper Functions**:
|
|
```go
|
|
// RFC 9457 compliant error responses
|
|
httperror.ValidationError(w, errors map[string][]string, detail string)
|
|
httperror.ProblemBadRequest(w, detail string)
|
|
httperror.ProblemUnauthorized(w, detail string)
|
|
httperror.ProblemForbidden(w, detail string)
|
|
httperror.ProblemNotFound(w, detail string)
|
|
httperror.ProblemConflict(w, detail string)
|
|
httperror.ProblemTooManyRequests(w, detail string)
|
|
httperror.ProblemInternalServerError(w, detail string)
|
|
|
|
// Legacy format (backward compatibility)
|
|
httperror.BadRequest(w, message string)
|
|
httperror.Unauthorized(w, message string)
|
|
httperror.NotFound(w, message string)
|
|
httperror.InternalServerError(w, message string)
|
|
```
|
|
|
|
**Validation Error Pattern**:
|
|
|
|
When implementing validation in DTOs, return structured errors:
|
|
|
|
```go
|
|
// DTO Validation
|
|
type ValidationErrors struct {
|
|
Errors map[string][]string
|
|
}
|
|
|
|
func (v *ValidationErrors) Error() string {
|
|
// Implement error interface for logging
|
|
var messages []string
|
|
for field, errs := range v.Errors {
|
|
for _, err := range errs {
|
|
messages = append(messages, fmt.Sprintf("%s: %s", field, err))
|
|
}
|
|
}
|
|
return fmt.Sprintf("validation errors: %v", messages)
|
|
}
|
|
|
|
// In DTO Validate() method
|
|
func (r *RegisterRequest) Validate() error {
|
|
validationErrors := make(map[string][]string)
|
|
|
|
// Collect all validation errors
|
|
if err := validateEmail(r.Email); err != nil {
|
|
validationErrors["email"] = append(validationErrors["email"], err.Error())
|
|
}
|
|
if err := validatePassword(r.Password); err != nil {
|
|
validationErrors["password"] = append(validationErrors["password"], err.Error())
|
|
}
|
|
|
|
// Return structured errors if any exist
|
|
if len(validationErrors) > 0 {
|
|
return &ValidationErrors{Errors: validationErrors}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// In Handler
|
|
if err := req.Validate(); err != nil {
|
|
if validationErr, ok := err.(*dto.ValidationErrors); ok {
|
|
// Return RFC 9457 validation error
|
|
httperror.ValidationError(w, validationErr.Errors, "One or more validation errors occurred")
|
|
return
|
|
}
|
|
// Fallback for non-validation errors
|
|
httperror.ProblemBadRequest(w, err.Error())
|
|
return
|
|
}
|
|
```
|
|
|
|
**Benefits**:
|
|
- ✅ Standardized error format across all endpoints
|
|
- ✅ Machine-readable error responses for frontend parsing
|
|
- ✅ Multiple errors returned at once (better UX)
|
|
- ✅ Field-specific error mapping for forms
|
|
- ✅ Industry-standard format (used by GitHub, Stripe, etc.)
|
|
- ✅ Proper Content-Type: `application/problem+json`
|
|
|
|
**Legacy Error Handling**:
|
|
|
|
For backward compatibility, legacy error functions are still available but RFC 9457 format is preferred for all new code:
|
|
|
|
**Domain errors**: Define in entity files
|
|
```go
|
|
var (
|
|
ErrUserNotFound = errors.New("user not found")
|
|
ErrEmailRequired = errors.New("email is required")
|
|
)
|
|
```
|
|
|
|
**Repository errors**: Wrap with context
|
|
```go
|
|
if err := query.Scan(...); err != nil {
|
|
return nil, fmt.Errorf("failed to get user: %w", err)
|
|
}
|
|
```
|
|
|
|
**Handler errors**: Use RFC 9457 format (preferred) or legacy format
|
|
```go
|
|
// Preferred: RFC 9457 format
|
|
httperror.ProblemBadRequest(w, "Invalid request body")
|
|
httperror.ProblemUnauthorized(w, "Missing tenant")
|
|
httperror.ProblemNotFound(w, "User not found")
|
|
httperror.ProblemInternalServerError(w, "Failed to create user")
|
|
|
|
// Legacy format (backward compatibility)
|
|
httperror.BadRequest(w, "invalid request body")
|
|
httperror.Unauthorized(w, "missing tenant")
|
|
httperror.NotFound(w, "user not found")
|
|
httperror.InternalServerError(w, "failed to create user")
|
|
```
|
|
|
|
### Logging
|
|
|
|
**CRITICAL: PII Redaction Requirements** 🔒
|
|
|
|
**CWE-532: Insertion of Sensitive Information into Log File**
|
|
|
|
MaplePress implements comprehensive PII (Personally Identifiable Information) redaction to comply with GDPR and security best practices. You **MUST NEVER** log actual emails, IP addresses, or other sensitive data in plaintext.
|
|
|
|
**Prohibited in Logs (NEVER log these directly):**
|
|
- ❌ Email addresses (plaintext)
|
|
- ❌ IP addresses (plaintext)
|
|
- ❌ Passwords (even hashed)
|
|
- ❌ API keys
|
|
- ❌ Session tokens
|
|
- ❌ Phone numbers
|
|
- ❌ Personal names (in most contexts)
|
|
- ❌ Payment information
|
|
|
|
**Required: Use Logger Helper Functions**
|
|
|
|
MaplePress provides secure logging helpers in `pkg/logger/`:
|
|
|
|
```go
|
|
import (
|
|
"go.uber.org/zap"
|
|
"codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/pkg/logger"
|
|
)
|
|
|
|
// ✅ CORRECT: Redacted email logging
|
|
h.logger.Info("user registered successfully",
|
|
zap.String("user_id", output.UserID),
|
|
zap.String("tenant_id", output.TenantID),
|
|
logger.EmailHash(output.UserEmail), // SHA-256 hash for correlation
|
|
logger.SafeEmail("email", output.UserEmail)) // Partial: "te***@ex***.com"
|
|
|
|
// ✅ CORRECT: Redacted tenant slug logging
|
|
h.logger.Warn("tenant slug already exists",
|
|
logger.TenantSlugHash(input.TenantSlug), // SHA-256 hash
|
|
logger.SafeTenantSlug("slug", input.TenantSlug)) // Partial: "my-***-tenant"
|
|
|
|
// ❌ WRONG: Plaintext email (NEVER DO THIS!)
|
|
h.logger.Info("user registered", zap.String("email", email)) // VIOLATION!
|
|
|
|
// ❌ WRONG: Plaintext IP address (NEVER DO THIS!)
|
|
h.logger.Info("request from", zap.String("ip", ipAddress)) // VIOLATION!
|
|
```
|
|
|
|
**Available Logger Helpers:**
|
|
|
|
**File**: `pkg/logger/sanitizer.go`
|
|
|
|
```go
|
|
// EmailHash - Returns SHA-256 hash of email for correlation
|
|
// Use for: Tracking user actions across logs without exposing email
|
|
logger.EmailHash(email string) zap.Field
|
|
// Example: logger.EmailHash("test@example.com")
|
|
// Output: "email_hash": "973dfe463ec85785f5f95af5ba3906ee..."
|
|
|
|
// SafeEmail - Returns partially redacted email
|
|
// Use for: Human-readable logs while protecting privacy
|
|
logger.SafeEmail(key string, email string) zap.Field
|
|
// Example: logger.SafeEmail("email_redacted", "test@example.com")
|
|
// Output: "email_redacted": "te***@ex***.com"
|
|
|
|
// TenantSlugHash - Returns SHA-256 hash of tenant slug
|
|
// Use for: Correlation without exposing tenant slug
|
|
logger.TenantSlugHash(slug string) zap.Field
|
|
// Example: logger.TenantSlugHash("my-company")
|
|
// Output: "tenant_slug_hash": "8f3d7e9a..."
|
|
|
|
// SafeTenantSlug - Returns partially redacted tenant slug
|
|
// Use for: Human-readable tenant references
|
|
logger.SafeTenantSlug(key string, slug string) zap.Field
|
|
// Example: logger.SafeTenantSlug("tenant_slug_redacted", "my-company")
|
|
// Output: "tenant_slug_redacted": "my-***-pany"
|
|
```
|
|
|
|
**IP Address Logging:**
|
|
|
|
IP addresses are **encrypted** before storage and should NEVER be logged in plaintext:
|
|
|
|
```go
|
|
// ✅ CORRECT: Log event without IP
|
|
h.logger.Info("user registered successfully",
|
|
zap.String("user_id", userID),
|
|
zap.String("tenant_id", tenantID))
|
|
// IP is encrypted and stored in database, not logged
|
|
|
|
// ❌ WRONG: Logging plaintext IP
|
|
h.logger.Info("registration from IP", zap.String("ip", ipAddress)) // VIOLATION!
|
|
```
|
|
|
|
**Comprehensive Logging Example:**
|
|
|
|
```go
|
|
// Success case - redacted PII
|
|
h.logger.Info("user registered successfully",
|
|
zap.String("user_id", output.UserID), // Safe: UUID
|
|
zap.String("tenant_id", output.TenantID), // Safe: UUID
|
|
logger.EmailHash(output.UserEmail)) // Safe: Hash
|
|
|
|
// Error case - redacted PII
|
|
h.logger.Error("failed to register user",
|
|
zap.Error(err), // Safe: Error message
|
|
logger.EmailHash(req.Email), // Safe: Hash for correlation
|
|
logger.SafeEmail("email_redacted", req.Email), // Safe: Partial email
|
|
logger.TenantSlugHash(req.TenantSlug), // Safe: Hash
|
|
logger.SafeTenantSlug("tenant_slug_redacted", req.TenantSlug)) // Safe: Partial
|
|
|
|
// Security event - no PII needed
|
|
h.logger.Warn("rate limit exceeded",
|
|
zap.String("path", r.URL.Path), // Safe: Public path
|
|
zap.String("method", r.Method)) // Safe: HTTP method
|
|
// Note: IP is extracted securely but not logged
|
|
```
|
|
|
|
**What CAN Be Logged Safely:**
|
|
- ✅ UUIDs (user_id, tenant_id, site_id)
|
|
- ✅ Email hashes (SHA-256)
|
|
- ✅ Partial emails (redacted)
|
|
- ✅ Tenant slug hashes
|
|
- ✅ Error messages (without PII)
|
|
- ✅ Request paths
|
|
- ✅ HTTP methods
|
|
- ✅ Status codes
|
|
- ✅ Timestamps
|
|
- ✅ Operation names
|
|
|
|
**Log Levels:**
|
|
|
|
Use appropriate log levels for different scenarios:
|
|
|
|
```go
|
|
// DEBUG - Development debugging (disabled in production)
|
|
h.logger.Debug("processing request",
|
|
zap.String("operation", "create_user"))
|
|
|
|
// INFO - Normal operations
|
|
h.logger.Info("user created successfully",
|
|
zap.String("user_id", userID))
|
|
|
|
// WARN - Recoverable issues, validation failures
|
|
h.logger.Warn("validation failed",
|
|
zap.Error(err))
|
|
|
|
// ERROR - System errors, failures
|
|
h.logger.Error("failed to save to database",
|
|
zap.Error(err),
|
|
zap.String("operation", "create_user"))
|
|
```
|
|
|
|
**Audit Trail:**
|
|
|
|
For audit purposes, sensitive data is stored **encrypted** in the database with the entity:
|
|
- IP addresses: Encrypted with AES-GCM before storage
|
|
- Timestamps: Stored with `CreatedFromIPTimestamp`, `ModifiedFromIPTimestamp`
|
|
- User actions: Tracked via `CreatedByUserID`, `ModifiedByUserID`
|
|
|
|
**Compliance:**
|
|
- GDPR Article 5(1)(f): Security of processing
|
|
- CWE-532: Insertion of Sensitive Information into Log File
|
|
- OWASP Logging Cheat Sheet compliance
|
|
|
|
**Remember:**
|
|
1. Always use `logger.EmailHash()` for email correlation
|
|
2. Use `logger.SafeEmail()` for human-readable partial emails
|
|
3. Never log IP addresses in plaintext
|
|
4. Never log passwords (even hashed)
|
|
5. Never log API keys or tokens
|
|
6. Use UUIDs for entity references (safe to log)
|
|
7. When in doubt, don't log it!
|
|
|
|
---
|
|
|
|
## Testing Guidelines
|
|
|
|
### Unit Tests
|
|
|
|
**Test domain validation**:
|
|
```go
|
|
func TestUser_Validate(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
user *User
|
|
wantErr error
|
|
}{
|
|
{
|
|
name: "valid user",
|
|
user: &User{Email: "test@example.com", Name: "Test"},
|
|
wantErr: nil,
|
|
},
|
|
{
|
|
name: "missing email",
|
|
user: &User{Name: "Test"},
|
|
wantErr: ErrEmailRequired,
|
|
},
|
|
}
|
|
// ... run tests
|
|
}
|
|
```
|
|
|
|
### Integration Tests
|
|
|
|
**Test repository with real Cassandra** (use Docker for tests):
|
|
```go
|
|
func TestRepository_Create(t *testing.T) {
|
|
// Setup: Start Cassandra container
|
|
// Create test session
|
|
// Apply schema
|
|
|
|
repo := NewRepository(session, logger)
|
|
|
|
user := &User{
|
|
ID: uuid.New().String(),
|
|
Email: "test@example.com",
|
|
// ...
|
|
}
|
|
|
|
err := repo.Create(context.Background(), "tenant-123", user)
|
|
assert.NoError(t, err)
|
|
|
|
// Verify in all tables
|
|
// ...
|
|
}
|
|
```
|
|
|
|
### Handler Tests
|
|
|
|
**Use httptest**:
|
|
```go
|
|
func TestCreateHandler_Handle(t *testing.T) {
|
|
// Create mock service
|
|
// Create handler
|
|
|
|
body := `{"email":"test@example.com","name":"Test"}`
|
|
req := httptest.NewRequest("POST", "/api/v1/users", strings.NewReader(body))
|
|
req.Header.Set("X-Tenant-ID", "tenant-123")
|
|
req.Header.Set("Content-Type", "application/json")
|
|
|
|
w := httptest.NewRecorder()
|
|
handler.Handle(w, req)
|
|
|
|
assert.Equal(t, http.StatusCreated, w.Code)
|
|
// ... verify response
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Common Pitfalls
|
|
|
|
### ❌ Forgetting tenant_id
|
|
|
|
**Wrong**:
|
|
```go
|
|
query := `SELECT id, email FROM users_by_id WHERE id = ?`
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
query := `SELECT tenant_id, id, email FROM users_by_id WHERE tenant_id = ? AND id = ?`
|
|
```
|
|
|
|
### ❌ Not using batched writes
|
|
|
|
**Wrong**:
|
|
```go
|
|
// Separate queries - not atomic!
|
|
db.Query("INSERT INTO users_by_id ...").Exec()
|
|
db.Query("INSERT INTO users_by_email ...").Exec()
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
batch := session.NewBatch(gocql.LoggedBatch)
|
|
batch.Query("INSERT INTO users_by_id ...")
|
|
batch.Query("INSERT INTO users_by_email ...")
|
|
session.ExecuteBatch(batch)
|
|
```
|
|
|
|
### ❌ Missing table model conversion
|
|
|
|
**Wrong**:
|
|
```go
|
|
// Using domain entity directly with database
|
|
db.Query("INSERT INTO users_by_id ...").Bind(user)
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
// Convert to table model first
|
|
userByID := models.FromUser(tenantID, user)
|
|
db.Query("INSERT INTO users_by_id ...").Bind(userByID)
|
|
```
|
|
|
|
### ❌ Import cycles
|
|
|
|
**Wrong**:
|
|
```
|
|
main → cmd/daemon → main (CYCLE!)
|
|
```
|
|
|
|
**Correct**:
|
|
```
|
|
main → cmd/daemon → app (no cycle)
|
|
```
|
|
|
|
Keep `InitializeApplication` in `app/` package, not `main`.
|
|
|
|
### ❌ Violating dependency rule
|
|
|
|
**Wrong**:
|
|
```go
|
|
// Domain layer importing repository
|
|
package domain
|
|
|
|
import "internal/repository/user" // WRONG!
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
// Domain defines interface, repository implements it
|
|
package domain
|
|
|
|
type Repository interface {
|
|
Create(...) error
|
|
}
|
|
```
|
|
|
|
### ❌ Hardcoded values
|
|
|
|
**Wrong**:
|
|
```go
|
|
tenantID := "default-tenant" // WRONG!
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
tenantID, err := middleware.GetTenantID(ctx)
|
|
```
|
|
|
|
### ❌ Not checking quotas before operations
|
|
|
|
**Wrong**:
|
|
```go
|
|
// Sync pages without checking quota
|
|
service.SyncPages(ctx, siteID, pages)
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
// Check quota first
|
|
if site.MonthlyPagesIndexed >= site.QuotaPages {
|
|
return httperror.Forbidden(w, "monthly indexing quota exceeded")
|
|
}
|
|
service.SyncPages(ctx, siteID, pages)
|
|
```
|
|
|
|
### ❌ Forgetting to hash API keys
|
|
|
|
**Wrong**:
|
|
```go
|
|
// Storing plaintext API key
|
|
site.APIKey = generatedKey
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
// Hash before storing
|
|
hash := security.HashAPIKey(generatedKey)
|
|
site.APIKeyHash = hash
|
|
site.APIKeyPrefix = generatedKey[:13]
|
|
site.APIKeyLastFour = generatedKey[len(generatedKey)-4:]
|
|
```
|
|
|
|
### ❌ Not using distributed mutex for quota updates
|
|
|
|
**Wrong**:
|
|
```go
|
|
// Race condition - multiple requests can exceed quota
|
|
site.RequestsCount++
|
|
repo.Update(site)
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
// Use distributed mutex
|
|
mutex.Lock(ctx, fmt.Sprintf("site:%s:quota", siteID))
|
|
defer mutex.Unlock()
|
|
site.RequestsCount++
|
|
repo.UpdateQuotas(site)
|
|
```
|
|
|
|
### ❌ Mixing authentication middleware
|
|
|
|
**Wrong**:
|
|
```go
|
|
// Using JWT middleware for plugin routes
|
|
mux.Handle("/api/v1/plugin/pages/sync",
|
|
jwtMiddleware.RequireAuth(syncHandler))
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
// Use API key middleware for plugin routes
|
|
mux.Handle("/api/v1/plugin/pages/sync",
|
|
apiKeyMiddleware.RequireAPIKey(syncHandler))
|
|
```
|
|
|
|
### ❌ Not deleting Meilisearch index when deleting site
|
|
|
|
**Wrong**:
|
|
```go
|
|
// Only delete from Cassandra
|
|
repo.Delete(ctx, tenantID, siteID)
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
// Delete from both Cassandra and Meilisearch
|
|
searchClient.DeleteIndex(ctx, site.SearchIndexName)
|
|
repo.Delete(ctx, tenantID, siteID)
|
|
```
|
|
|
|
### ❌ Forgetting to strip HTML from content
|
|
|
|
**Wrong**:
|
|
```go
|
|
// Indexing raw HTML content
|
|
page.Content = wordpressPage.Content
|
|
```
|
|
|
|
**Correct**:
|
|
```go
|
|
// Strip HTML tags before indexing
|
|
page.Content = stripHTML(wordpressPage.Content)
|
|
```
|
|
|
|
---
|
|
|
|
## Quick Reference
|
|
|
|
### Project Structure
|
|
```
|
|
cloud/maplepress-backend/
|
|
├── app/ # Application & Wire DI
|
|
│ ├── app.go
|
|
│ ├── wire.go
|
|
│ └── wire_gen.go (generated)
|
|
├── cmd/ # CLI commands
|
|
│ ├── daemon/
|
|
│ ├── root.go
|
|
│ └── version/
|
|
├── config/ # Configuration
|
|
├── internal/
|
|
│ ├── domain/ # Entities & interfaces
|
|
│ │ └── user/
|
|
│ │ ├── entity.go
|
|
│ │ └── repository.go
|
|
│ ├── repository/ # Data access
|
|
│ │ └── user/
|
|
│ │ ├── models/
|
|
│ │ │ ├── user_by_id.go
|
|
│ │ │ ├── user_by_email.go
|
|
│ │ │ └── user_by_date.go
|
|
│ │ ├── impl.go
|
|
│ │ ├── create.go
|
|
│ │ ├── get.go
|
|
│ │ └── schema.cql
|
|
│ ├── usecase/ # Business logic
|
|
│ │ └── user/
|
|
│ │ ├── create.go
|
|
│ │ └── get.go
|
|
│ ├── service/ # Orchestration
|
|
│ │ └── user_service.go
|
|
│ └── interface/http/ # HTTP layer
|
|
│ ├── dto/
|
|
│ │ └── user/
|
|
│ ├── handler/
|
|
│ │ ├── healthcheck/
|
|
│ │ └── user/
|
|
│ ├── middleware/
|
|
│ │ ├── tenant.go
|
|
│ │ └── logger.go
|
|
│ └── server.go
|
|
├── pkg/ # Infrastructure
|
|
│ ├── logger/
|
|
│ ├── storage/
|
|
│ │ ├── database/
|
|
│ │ └── cache/
|
|
│ └── httperror/
|
|
├── docker-compose.dev.yml
|
|
├── Taskfile.yml
|
|
├── .env.sample
|
|
└── main.go
|
|
```
|
|
|
|
### Common Commands
|
|
|
|
```bash
|
|
# Development
|
|
task dev # Start backend (auto-migrate + hot-reload)
|
|
task end # Stop backend
|
|
task console # Open bash in backend container
|
|
|
|
# Testing
|
|
task test # Run tests
|
|
|
|
# Code quality
|
|
task format # Format code with goimports
|
|
task lint # Run golint
|
|
task vet # Run go vet
|
|
task check # Run format + lint + vet
|
|
|
|
# Dependencies
|
|
task vendor # Download and vendor dependencies
|
|
task upgradelib # Update all Go libraries
|
|
|
|
# Database & Migrations
|
|
task db:clear # Clear database
|
|
task db:reset # Migration down + up
|
|
task migrate:up # Run migrations
|
|
task migrate:down # Rollback migrations
|
|
task migrate:create # Create new migration
|
|
|
|
# Manual operations (rarely needed)
|
|
task build # Build binary
|
|
task wire # Regenerate Wire DI code
|
|
|
|
# Deployment (DevOps)
|
|
task deploy # Build and push production container
|
|
task deployqa # Build and push QA container
|
|
|
|
# Cleanup
|
|
task dev-clean # Stop Docker and remove volumes
|
|
task clean # Clean build artifacts
|
|
```
|
|
|
|
### API Patterns
|
|
|
|
**Create Resource**:
|
|
```bash
|
|
POST /api/v1/users
|
|
Headers: X-Tenant-ID, Content-Type: application/json
|
|
Body: {"email": "...", "name": "..."}
|
|
Response: 201 Created
|
|
```
|
|
|
|
**Get Resource**:
|
|
```bash
|
|
GET /api/v1/users/{id}
|
|
Headers: X-Tenant-ID
|
|
Response: 200 OK
|
|
```
|
|
|
|
**Update Resource**:
|
|
```bash
|
|
PUT /api/v1/users/{id}
|
|
Headers: X-Tenant-ID, Content-Type: application/json
|
|
Body: {"name": "..."}
|
|
Response: 200 OK
|
|
```
|
|
|
|
**Delete Resource**:
|
|
```bash
|
|
DELETE /api/v1/users/{id}
|
|
Headers: X-Tenant-ID
|
|
Response: 204 No Content
|
|
```
|
|
|
|
---
|
|
|
|
## Getting Help
|
|
|
|
- **Architecture Questions**: Review this guide, README.md, and CLAUDE.md
|
|
- **Cassandra Questions**: Check migration files in `migrations/` and `schema.cql` files
|
|
- **API Questions**: See handler files in `internal/interface/http/handler/`
|
|
- **Configuration**: See `.env.sample` for all available options
|
|
- **Meilisearch**: Check `pkg/search/` for client implementation
|
|
- **Quota System**: Review `internal/domain/site/entity.go` and quota use cases
|
|
- **Authentication**: See `pkg/security/` for JWT and API key utilities
|
|
- **Middleware**: Check `internal/interface/http/middleware/` for auth patterns
|
|
|
|
---
|
|
|
|
## Key Takeaways
|
|
|
|
**MaplePress Backend is a production-ready, multi-tenant SaaS platform that demonstrates:**
|
|
|
|
1. **Clean Architecture** - Clear separation of concerns with dependency inversion
|
|
2. **Focused Use Cases** - Single-responsibility operations for composability
|
|
3. **Multi-Table Denormalization** - Optimized Cassandra access patterns
|
|
4. **Dual Authentication** - JWT for users, API keys for WordPress plugins
|
|
5. **Comprehensive Quota System** - Cumulative storage + monthly quotas with cron resets
|
|
6. **Meilisearch Integration** - Fast, typo-tolerant full-text search
|
|
7. **Wire Dependency Injection** - Compile-time safety and clarity
|
|
8. **Production-Ready Security** - Argon2id passwords, SHA-256 API key hashing, distributed locking
|
|
|
|
**Remember**:
|
|
- Always include `tenant_id` in Cassandra queries
|
|
- Use batched writes for multi-table operations
|
|
- Check quotas before resource-intensive operations
|
|
- Use distributed mutex for concurrent quota updates
|
|
- Hash API keys (SHA-256) before storage
|
|
- Strip HTML from content before indexing
|
|
- Use appropriate middleware (JWT vs API key)
|
|
- Delete Meilisearch indexes when deleting sites
|
|
- Follow the dependency rule (dependencies point inward)
|
|
- Keep explicit table models for Cassandra
|
|
- Break use cases into focused, single-responsibility operations
|
|
- Test your code thoroughly
|
|
|
|
**The architecture shows evolutionary refinement** with the new focused use case pattern and dual repository approach, demonstrating thoughtful migration toward better patterns while maintaining backward compatibility.
|
|
|
|
Happy coding! 🚀
|