Initial commit: Open sourcing all of the Maple Open Technologies code.

cloud/maplepress-backend/pkg/distributedmutex/README.md

@@ -0,0 +1,237 @@
+# Distributed Mutex
+
+A Redis-based distributed mutex implementation for coordinating access to shared resources across multiple application instances.
+
+## Overview
+
+This package provides a distributed locking mechanism using Redis as the coordination backend. It's built on top of the `redislock` library and provides a simple interface for acquiring and releasing locks across distributed systems.
+
+## Features
+
+- **Distributed Locking**: Coordinate access to shared resources across multiple application instances
+- **Automatic Retry**: Built-in retry logic with configurable backoff strategy
+- **Thread-Safe**: Safe for concurrent use within a single application
+- **Formatted Keys**: Support for formatted lock keys using `Acquiref` and `Releasef`
+- **Logging**: Integrated zap logging for debugging and monitoring
+
+## Installation
+
+The package is already included in the project. The required dependency (`github.com/bsm/redislock`) is automatically installed.
+
+## Interface
+
+```go
+type Adapter interface {
+    Acquire(ctx context.Context, key string)
+    Acquiref(ctx context.Context, format string, a ...any)
+    Release(ctx context.Context, key string)
+    Releasef(ctx context.Context, format string, a ...any)
+}
+```
+
+## Usage
+
+### Basic Example
+
+```go
+import (
+    "context"
+    "github.com/redis/go-redis/v9"
+    "go.uber.org/zap"
+    "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/pkg/distributedmutex"
+)
+
+// Create Redis client
+redisClient := redis.NewClient(&redis.Options{
+    Addr: "localhost:6379",
+})
+
+// Create logger
+logger, _ := zap.NewProduction()
+
+// Create distributed mutex adapter
+mutex := distributedmutex.NewAdapter(logger, redisClient)
+
+// Acquire a lock
+ctx := context.Background()
+mutex.Acquire(ctx, "my-resource-key")
+
+// ... perform operations on the protected resource ...
+
+// Release the lock
+mutex.Release(ctx, "my-resource-key")
+```
+
+### Formatted Keys Example
+
+```go
+// Acquire lock with formatted key
+tenantID := "tenant-123"
+resourceID := "resource-456"
+
+mutex.Acquiref(ctx, "tenant:%s:resource:%s", tenantID, resourceID)
+
+// ... perform operations ...
+
+mutex.Releasef(ctx, "tenant:%s:resource:%s", tenantID, resourceID)
+```
+
+### Integration with Dependency Injection (Wire)
+
+```go
+// In your Wire provider set
+wire.NewSet(
+    distributedmutex.ProvideDistributedMutexAdapter,
+    // ... other providers
+)
+
+// Use in your application
+func NewMyService(mutex distributedmutex.Adapter) *MyService {
+    return &MyService{
+        mutex: mutex,
+    }
+}
+```
+
+## Configuration
+
+### Lock Duration
+
+The default lock duration is **1 minute**. Locks are automatically released after this time to prevent deadlocks.
+
+### Retry Strategy
+
+- **Retry Interval**: 250ms
+- **Max Retries**: 20 attempts
+- **Total Max Wait Time**: ~5 seconds (20 × 250ms)
+
+If a lock cannot be obtained after all retries, an error is logged and the `Acquire` method returns without blocking indefinitely.
+
+## Best Practices
+
+1. **Always Release Locks**: Ensure locks are released even in error cases using `defer`
+   ```go
+   mutex.Acquire(ctx, "my-key")
+   defer mutex.Release(ctx, "my-key")
+   ```
+
+2. **Use Descriptive Keys**: Use clear, hierarchical key names
+   ```go
+   // Good
+   mutex.Acquire(ctx, "tenant:123:user:456:update")
+
+   // Not ideal
+   mutex.Acquire(ctx, "lock1")
+   ```
+
+3. **Keep Critical Sections Short**: Minimize the time locks are held to improve concurrency
+
+4. **Handle Timeouts**: Use context with timeout for critical operations
+   ```go
+   ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+   defer cancel()
+   mutex.Acquire(ctx, "my-key")
+   ```
+
+5. **Avoid Nested Locks**: Be careful with acquiring multiple locks to avoid deadlocks
+
+## Logging
+
+The adapter logs the following events:
+
+- **Debug**: Lock acquisition and release operations
+- **Error**: Failed lock acquisitions, timeout errors, and release failures
+- **Warn**: Attempts to release non-existent locks
+
+## Thread Safety
+
+The adapter is safe for concurrent use within a single application instance. It uses an internal mutex to protect the lock instances map from concurrent access by multiple goroutines.
+
+## Error Handling
+
+The current implementation logs errors but does not return them. Consider this when using the adapter:
+
+- Lock acquisition failures are logged but don't panic
+- The application continues running even if locks fail
+- Check logs for lock-related issues in production
+
+## Limitations
+
+1. **Lock Duration**: Locks automatically expire after 1 minute
+2. **No Lock Extension**: Currently doesn't support extending lock duration
+3. **No Deadlock Detection**: Manual deadlock prevention is required
+4. **Redis Dependency**: Requires a running Redis instance
+
+## Example Use Cases
+
+### Preventing Duplicate Processing
+
+```go
+func ProcessJob(ctx context.Context, jobID string, mutex distributedmutex.Adapter) {
+    lockKey := fmt.Sprintf("job:processing:%s", jobID)
+
+    mutex.Acquire(ctx, lockKey)
+    defer mutex.Release(ctx, lockKey)
+
+    // Process job - guaranteed only one instance processes this job
+    // ...
+}
+```
+
+### Coordinating Resource Updates
+
+```go
+func UpdateTenantSettings(ctx context.Context, tenantID string, mutex distributedmutex.Adapter) error {
+    mutex.Acquiref(ctx, "tenant:%s:settings:update", tenantID)
+    defer mutex.Releasef(ctx, "tenant:%s:settings:update", tenantID)
+
+    // Safe to update tenant settings
+    // ...
+    return nil
+}
+```
+
+### Rate Limiting Operations
+
+```go
+func RateLimitedOperation(ctx context.Context, userID string, mutex distributedmutex.Adapter) {
+    lockKey := fmt.Sprintf("ratelimit:user:%s", userID)
+
+    mutex.Acquire(ctx, lockKey)
+    defer mutex.Release(ctx, lockKey)
+
+    // Perform rate-limited operation
+    // ...
+}
+```
+
+## Troubleshooting
+
+### Lock Not Acquired
+
+**Problem**: Locks are not being acquired (error in logs)
+
+**Solutions**:
+- Verify Redis is running and accessible
+- Check network connectivity to Redis
+- Ensure Redis has sufficient memory
+- Check for Redis errors in logs
+
+### Lock Contention
+
+**Problem**: Frequent lock acquisition failures due to contention
+
+**Solutions**:
+- Reduce critical section duration
+- Use more specific lock keys to reduce contention
+- Consider increasing retry limits if appropriate
+- Review application architecture for excessive locking
+
+### Memory Leaks
+
+**Problem**: Lock instances accumulating in memory
+
+**Solutions**:
+- Ensure all `Acquire` calls have corresponding `Release` calls
+- Use `defer` to guarantee lock release
+- Monitor lock instance map size in production