# Distributed Mutex A Redis-based distributed mutex implementation for coordinating access to shared resources across multiple application instances. ## Overview This package provides a distributed locking mechanism using Redis as the coordination backend. It's built on top of the `redislock` library and provides a simple interface for acquiring and releasing locks across distributed systems. ## Features - **Distributed Locking**: Coordinate access to shared resources across multiple application instances - **Automatic Retry**: Built-in retry logic with configurable backoff strategy - **Thread-Safe**: Safe for concurrent use within a single application - **Formatted Keys**: Support for formatted lock keys using `Acquiref` and `Releasef` - **Logging**: Integrated zap logging for debugging and monitoring ## Installation The package is already included in the project. The required dependency (`github.com/bsm/redislock`) is automatically installed. ## Interface ```go type Adapter interface { Acquire(ctx context.Context, key string) Acquiref(ctx context.Context, format string, a ...any) Release(ctx context.Context, key string) Releasef(ctx context.Context, format string, a ...any) } ``` ## Usage ### Basic Example ```go import ( "context" "github.com/redis/go-redis/v9" "go.uber.org/zap" "codeberg.org/mapleopentech/monorepo/cloud/maplepress-backend/pkg/distributedmutex" ) // Create Redis client redisClient := redis.NewClient(&redis.Options{ Addr: "localhost:6379", }) // Create logger logger, _ := zap.NewProduction() // Create distributed mutex adapter mutex := distributedmutex.NewAdapter(logger, redisClient) // Acquire a lock ctx := context.Background() mutex.Acquire(ctx, "my-resource-key") // ... perform operations on the protected resource ... // Release the lock mutex.Release(ctx, "my-resource-key") ``` ### Formatted Keys Example ```go // Acquire lock with formatted key tenantID := "tenant-123" resourceID := "resource-456" mutex.Acquiref(ctx, "tenant:%s:resource:%s", tenantID, resourceID) // ... perform operations ... mutex.Releasef(ctx, "tenant:%s:resource:%s", tenantID, resourceID) ``` ### Integration with Dependency Injection (Wire) ```go // In your Wire provider set wire.NewSet( distributedmutex.ProvideDistributedMutexAdapter, // ... other providers ) // Use in your application func NewMyService(mutex distributedmutex.Adapter) *MyService { return &MyService{ mutex: mutex, } } ``` ## Configuration ### Lock Duration The default lock duration is **1 minute**. Locks are automatically released after this time to prevent deadlocks. ### Retry Strategy - **Retry Interval**: 250ms - **Max Retries**: 20 attempts - **Total Max Wait Time**: ~5 seconds (20 × 250ms) If a lock cannot be obtained after all retries, an error is logged and the `Acquire` method returns without blocking indefinitely. ## Best Practices 1. **Always Release Locks**: Ensure locks are released even in error cases using `defer` ```go mutex.Acquire(ctx, "my-key") defer mutex.Release(ctx, "my-key") ``` 2. **Use Descriptive Keys**: Use clear, hierarchical key names ```go // Good mutex.Acquire(ctx, "tenant:123:user:456:update") // Not ideal mutex.Acquire(ctx, "lock1") ``` 3. **Keep Critical Sections Short**: Minimize the time locks are held to improve concurrency 4. **Handle Timeouts**: Use context with timeout for critical operations ```go ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel() mutex.Acquire(ctx, "my-key") ``` 5. **Avoid Nested Locks**: Be careful with acquiring multiple locks to avoid deadlocks ## Logging The adapter logs the following events: - **Debug**: Lock acquisition and release operations - **Error**: Failed lock acquisitions, timeout errors, and release failures - **Warn**: Attempts to release non-existent locks ## Thread Safety The adapter is safe for concurrent use within a single application instance. It uses an internal mutex to protect the lock instances map from concurrent access by multiple goroutines. ## Error Handling The current implementation logs errors but does not return them. Consider this when using the adapter: - Lock acquisition failures are logged but don't panic - The application continues running even if locks fail - Check logs for lock-related issues in production ## Limitations 1. **Lock Duration**: Locks automatically expire after 1 minute 2. **No Lock Extension**: Currently doesn't support extending lock duration 3. **No Deadlock Detection**: Manual deadlock prevention is required 4. **Redis Dependency**: Requires a running Redis instance ## Example Use Cases ### Preventing Duplicate Processing ```go func ProcessJob(ctx context.Context, jobID string, mutex distributedmutex.Adapter) { lockKey := fmt.Sprintf("job:processing:%s", jobID) mutex.Acquire(ctx, lockKey) defer mutex.Release(ctx, lockKey) // Process job - guaranteed only one instance processes this job // ... } ``` ### Coordinating Resource Updates ```go func UpdateTenantSettings(ctx context.Context, tenantID string, mutex distributedmutex.Adapter) error { mutex.Acquiref(ctx, "tenant:%s:settings:update", tenantID) defer mutex.Releasef(ctx, "tenant:%s:settings:update", tenantID) // Safe to update tenant settings // ... return nil } ``` ### Rate Limiting Operations ```go func RateLimitedOperation(ctx context.Context, userID string, mutex distributedmutex.Adapter) { lockKey := fmt.Sprintf("ratelimit:user:%s", userID) mutex.Acquire(ctx, lockKey) defer mutex.Release(ctx, lockKey) // Perform rate-limited operation // ... } ``` ## Troubleshooting ### Lock Not Acquired **Problem**: Locks are not being acquired (error in logs) **Solutions**: - Verify Redis is running and accessible - Check network connectivity to Redis - Ensure Redis has sufficient memory - Check for Redis errors in logs ### Lock Contention **Problem**: Frequent lock acquisition failures due to contention **Solutions**: - Reduce critical section duration - Use more specific lock keys to reduce contention - Consider increasing retry limits if appropriate - Review application architecture for excessive locking ### Memory Leaks **Problem**: Lock instances accumulating in memory **Solutions**: - Ensure all `Acquire` calls have corresponding `Release` calls - Use `defer` to guarantee lock release - Monitor lock instance map size in production