Bartlomiej Mika 755d54a99d Initial commit: Open sourcing all of the Maple Open Technologies code.

2025-12-02 14:33:08 -05:00

17 KiB

Raw Blame History

Site Verification System

Overview

MaplePress implements DNS-based domain ownership verification to ensure users actually own the domains they register. Sites start in "pending" status and remain there until verified through DNS TXT record validation.

Verification Method: DNS TXT Records

MaplePress uses DNS TXT record verification - the industry standard used by Google, Cloudflare, and other major services. This proves domain ownership, not just dashboard access.

Why DNS Verification?

Proves domain ownership: Only someone with DNS access can add TXT records
Industry standard: Same method used by Google Search Console, Cloudflare, etc.
Secure: Cannot be spoofed or bypassed without actual domain control
Automatic: Backend performs verification via DNS lookup

Site Status Lifecycle

Status Constants

File: internal/domain/site/site.go:61-67

const (
    StatusPending   = "pending"   // Site created, awaiting DNS verification
    StatusActive    = "active"    // Site verified via DNS and operational
    StatusInactive  = "inactive"  // User temporarily disabled
    StatusSuspended = "suspended" // Suspended due to violation or non-payment
    StatusArchived  = "archived"  // Soft deleted
)

1. Site Creation (Pending State)

File: internal/usecase/site/create.go

When a site is created via POST /api/v1/sites:

What Gets Generated

API Key (test or live mode)
- Test mode: test_sk_... (skips DNS verification)
- Live mode: live_sk_... (requires DNS verification)
Verification Token (lines 88-92)
- Format: mvp_ + 128-bit random token (base64-encoded)
- Example: mvp_xyz789abc123
- Used in DNS TXT record: maplepress-verify={token}
DNS Verification Instructions
- Provides step-by-step DNS setup guide
- Includes domain registrar examples (GoDaddy, Namecheap, Cloudflare, etc.)
- Explains DNS propagation timing (5-10 minutes typical)
Site Entity (lines 104-113)
- Initial status: StatusPending
- IsVerified: false
- VerificationToken: Set to generated token

Response Example

{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "domain": "example.com",
  "site_url": "https://example.com",
  "api_key": "live_sk_a1b2...",           // ⚠️ SHOWN ONLY ONCE
  "verification_token": "mvp_xyz789abc123",
  "status": "pending",
  "search_index_name": "site_...",
  "verification_instructions": "To verify ownership of example.com, add this DNS TXT record:\n\nHost/Name: example.com\nType: TXT\nValue: maplepress-verify=mvp_xyz789abc123\n\nInstructions:\n1. Log in to your domain registrar...\n2. Find DNS settings...\n3. Add a new TXT record...\n4. Wait 5-10 minutes for DNS propagation\n5. Click 'Verify Domain' in MaplePress"
}

Documentation: docs/API/create-site.md

2. Test Mode Bypass

File: internal/domain/site/site.go:115-125

Test Mode Detection

func (s *Site) IsTestMode() bool {
    return len(s.APIKeyPrefix) >= 7 && s.APIKeyPrefix[:7] == "test_sk"
}

Verification Requirement Check

func (s *Site) RequiresVerification() bool {
    return !s.IsTestMode()  // Test mode sites skip verification
}

Key Points:

Sites with test_sk_ API keys skip verification entirely
Useful for development and testing
Test mode sites can sync pages immediately

3. API Access Control

File: internal/domain/site/site.go:127-140

CanAccessAPI() Method

func (s *Site) CanAccessAPI() bool {
    // Allow active sites (fully verified)
    if s.Status == StatusActive {
        return true
    }
    // Allow pending sites (waiting for verification) for initial setup
    if s.Status == StatusPending {
        return true
    }
    // Block inactive, suspended, or archived sites
    return false
}

Important: Pending sites CAN access the API for:

Status checks (GET /api/v1/plugin/status)
Initial plugin setup
Retrieving site information

4. Verification Enforcement

Where Verification is Required

File: internal/usecase/page/sync.go:85-89

When syncing pages (POST /api/v1/plugin/sync):

// Verify site is verified (skip for test mode)
if site.RequiresVerification() && !site.IsVerified {
    uc.logger.Warn("site not verified", zap.String("site_id", siteID.String()))
    return nil, domainsite.ErrSiteNotVerified
}

Error: internal/domain/site/errors.go:22

ErrSiteNotVerified = errors.New("site is not verified")

HTTP Response

{
  "type": "about:blank",
  "title": "Forbidden",
  "status": 403,
  "detail": "site is not verified"
}

5. DNS Verification Implementation

DNS Verifier Package

File: pkg/dns/verifier.go

type Verifier struct {
    resolver *net.Resolver
    logger   *zap.Logger
}

func (v *Verifier) VerifyDomainOwnership(ctx context.Context, domain string, expectedToken string) (bool, error) {
    // Create context with 10-second timeout
    lookupCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
    defer cancel()

    // Look up TXT records for the domain
    txtRecords, err := v.resolver.LookupTXT(lookupCtx, domain)
    if err != nil {
        return false, fmt.Errorf("DNS lookup failed: %w", err)
    }

    // Expected format: "maplepress-verify=TOKEN"
    expectedRecord := fmt.Sprintf("maplepress-verify=%s", expectedToken)

    for _, record := range txtRecords {
        if strings.TrimSpace(record) == expectedRecord {
            return true, nil // Domain ownership verified!
        }
    }

    return false, nil // TXT record not found
}

DNS Verification Use Case

File: internal/usecase/site/verify.go

The verification use case performs DNS lookup:

func (uc *VerifySiteUseCase) Execute(ctx context.Context, tenantID gocql.UUID, siteID gocql.UUID, input *VerifySiteInput) (*VerifySiteOutput, error) {
    // Get site from repository
    site, err := uc.repo.GetByID(ctx, tenantID, siteID)
    if err != nil {
        return nil, domainsite.ErrSiteNotFound
    }

    // Check if already verified
    if site.IsVerified {
        return &VerifySiteOutput{Success: true, Status: site.Status, Message: "Site is already verified"}, nil
    }

    // Test mode sites skip DNS verification
    if site.IsTestMode() {
        site.Verify()
        if err := uc.repo.Update(ctx, site); err != nil {
            return nil, fmt.Errorf("failed to update site: %w", err)
        }
        return &VerifySiteOutput{Success: true, Status: site.Status, Message: "Test mode site verified successfully"}, nil
    }

    // Perform DNS TXT record verification
    verified, err := uc.dnsVerifier.VerifyDomainOwnership(ctx, site.Domain, site.VerificationToken)
    if err != nil {
        return nil, fmt.Errorf("DNS verification failed: %w", err)
    }

    if !verified {
        return nil, fmt.Errorf("DNS TXT record not found. Please add the verification record to your domain's DNS settings")
    }

    // DNS verification successful - mark site as verified
    site.Verify()
    if err := uc.repo.Update(ctx, site); err != nil {
        return nil, fmt.Errorf("failed to update site: %w", err)
    }

    return &VerifySiteOutput{Success: true, Status: site.Status, Message: "Domain ownership verified successfully via DNS TXT record"}, nil
}

Verify Method

File: internal/domain/site/site.go:169-175

// Verify marks the site as verified
func (s *Site) Verify() {
    s.IsVerified = true
    s.Status = StatusActive
    s.VerificationToken = "" // Clear token after verification
    s.UpdatedAt = time.Now()
}

6. What Pending Sites Can Do

File: internal/interface/http/handler/plugin/status_handler.go

Allowed Operations

✅ GET /api/v1/plugin/status - Check site status and quotas

Returns full site details
Shows is_verified: false
Shows status: "pending"

Blocked Operations

❌ POST /api/v1/plugin/sync - Sync pages to search index

Returns 403 Forbidden
Error: "site is not verified"

❌ POST /api/v1/plugin/search - Perform searches

Blocked for unverified sites

❌ DELETE /api/v1/plugin/pages - Delete pages

Blocked for unverified sites

7. Verification Token Details

File: internal/usecase/site/generate_verification_token.go

Token Generation

func (uc *GenerateVerificationTokenUseCase) Execute() (string, error) {
    b := make([]byte, 16) // 16 bytes = 128 bits
    if _, err := rand.Read(b); err != nil {
        uc.logger.Error("failed to generate random bytes", zap.Error(err))
        return "", err
    }

    token := base64.RawURLEncoding.EncodeToString(b)
    verificationToken := "mvp_" + token // mvp = maplepress verify

    uc.logger.Info("verification token generated")
    return verificationToken, nil
}

Token Format:

Prefix: mvp_ (MaplePress Verify)
Encoding: Base64 URL-safe (no padding)
Strength: 128-bit cryptographic randomness
Example: mvp_dGhpc2lzYXRlc3Q

Security:

Never exposed in JSON responses (marked with json:"-")
Stored in database only
Cleared after verification

8. DNS Verification Flow

Step-by-Step Process

User creates site via dashboard (POST /api/v1/sites)
- Backend generates API key and verification token
- Site status: pending
- Response includes DNS setup instructions
- User receives: API key (once), verification token, DNS TXT record format
User adds DNS TXT record to domain registrar
- Logs in to domain registrar (GoDaddy, Namecheap, Cloudflare, etc.)
- Navigates to DNS management
- Adds TXT record: maplepress-verify={verification_token}
- Waits 5-10 minutes for DNS propagation (can take up to 48 hours)
User installs WordPress plugin
- Plugin activation screen shows
- User enters API key
- Plugin connects to backend
Plugin checks status (GET /api/v1/plugin/status)
- Backend returns site status: pending
- Plugin shows "Site not verified" message
- Plugin displays DNS instructions if not verified
User verifies site (POST /api/v1/sites/{id}/verify)
- User clicks "Verify Domain" in plugin or dashboard
- No request body needed (empty POST)
- Backend performs DNS TXT lookup for domain
- Backend checks for record: maplepress-verify={verification_token}
- If found: Site transitions pending → active, IsVerified set to true
- If not found: Returns error with DNS troubleshooting instructions
Plugin can now sync (POST /api/v1/plugin/sync)
- Verification check passes
- Pages are synced and indexed
- Search functionality enabled

9. Architectural Design Decisions

Why Pending Sites Can Access API

From site.go:127-140, the design allows pending sites to:

Check their status
View usage statistics
Prepare for verification

This is a deliberate UX decision to allow:

Plugin to be activated immediately
Admin to see connection status
Admin to complete verification steps
Smoother onboarding experience

Why DNS Verification is Required

DNS verification prevents:

Domain squatting: Claiming domains you don't own
Abuse: Indexing content from sites you don't control
Impersonation: Pretending to be another site
Unauthorized access: Using the service without permission

DNS TXT record verification is the industry standard because:

Proves domain control: Only someone with DNS access can add TXT records
Widely recognized: Same method used by Google Search Console, Cloudflare, etc.
Cannot be spoofed: Requires actual access to domain registrar
Automatic verification: Backend can verify ownership without manual review

Test Mode Rationale

Test mode (test_sk_ keys) bypasses verification to enable:

Local development without DNS
Integration testing in CI/CD
Staging environments
Development workflows

10. Security Considerations

Token Security

Generation:
- Cryptographically secure random generation
- 128-bit entropy (sufficient for this use case)
- Base64 URL-safe encoding
Storage:
- Stored in database as plain text (used in DNS TXT record)
- Cleared after successful verification
- Only accessible to authenticated tenant
DNS Verification Security:
- DNS TXT records are public (as intended)
- Token is meaningless without backend verification
- 10-second timeout on DNS lookups prevents DoS
- Token cleared after verification prevents reuse

Attack Vectors Mitigated

Domain Squatting: DNS verification proves domain ownership
Token Guessing: 128-bit entropy makes brute force infeasible
Token Reuse: Token cleared after successful verification
Man-in-the-Middle: HTTPS required for all API calls
DNS Spoofing: Uses multiple DNS resolvers and validates responses
DNS Cache Poisoning: 10-second timeout limits attack window

11. API Documentation

See individual endpoint documentation:

Create Site - Initial site creation
Verify Site - Site verification endpoint
Plugin Status - Check verification status
Sync Pages - Requires verification

12. WordPress Plugin Integration

The WordPress plugin should:

On Activation:
- Prompt user for API key
- Connect to backend
- Check verification status
If Not Verified:
- Display DNS TXT record instructions
- Show the exact TXT record to add: maplepress-verify={token}
- Provide domain registrar examples (GoDaddy, Namecheap, Cloudflare)
- Explain DNS propagation timing (5-10 minutes)
- Provide "Verify Domain" button
- Disable sync/search features
Verification Process:
- User clicks "Verify Domain"
- Plugin calls POST /api/v1/sites/{id}/verify (no body)
- Backend performs DNS TXT lookup
- If successful: Enable all features
- If failed: Show specific DNS error (record not found, timeout, etc.)
After Verification:
- Enable all features
- Allow page synchronization
- Enable search functionality
- Hide verification prompts
Error Handling:
- Handle 403 "site is not verified" gracefully
- Guide user to DNS verification process
- Show DNS troubleshooting tips (check propagation, verify record format)
- Retry verification status check

13. Database Schema

Site Table Fields

sites_by_id:
  - id (UUID, primary key)
  - tenant_id (UUID)
  - status (text: pending|active|inactive|suspended|archived)
  - is_verified (boolean)
  - verification_token (text, sensitive)
  - ...

Indexes Required

No special indexes needed for DNS verification - uses existing site_id and tenant_id lookups.

14. Troubleshooting

Common Issues

DNS TXT record not found:
- Check DNS propagation status (use dig or nslookup)
- Verify record format: maplepress-verify={exact_token}
- Wait 5-10 minutes for DNS propagation
- Check that TXT record was added to correct domain/subdomain
- Verify no typos in the verification token
DNS lookup timeout:
- Check domain's DNS servers are responding
- Verify domain is properly registered
- Check for DNS configuration issues
- Try again after DNS stabilizes
Site stuck in pending:
- Verify DNS TXT record is correctly set
- Call verification endpoint: POST /api/v1/sites/{id}/verify
- Check logs for DNS lookup errors
- Use DNS checking tools (dig, nslookup) to verify record
Test mode not working:
- Verify API key starts with test_sk_
- Check IsTestMode() logic in site.go:115-125
- Test mode sites skip DNS verification entirely
DNS verification fails:
- Token may have been cleared (already verified)
- DNS record format incorrect
- Wrong domain or subdomain
- Check error logs for specific DNS errors

Debug Commands

# Check DNS TXT record manually
dig TXT example.com
nslookup -type=TXT example.com

# Check site status
curl -X GET http://localhost:8000/api/v1/sites/{id} \
  -H "Authorization: JWT {token}"

# Verify site via DNS
curl -X POST http://localhost:8000/api/v1/sites/{id}/verify \
  -H "Authorization: JWT {token}"

15. Future Enhancements

Potential improvements to the verification system:

Token Expiration:
- Add 24-48 hour expiration for verification tokens
- Allow token regeneration
- Email token to site admin
Alternative Verification Methods:
- Meta tag verification (alternative to DNS)
- File upload verification (.well-known/maplepress-verify.txt)
- WordPress plugin automatic verification (callback endpoint)
Automatic Re-verification:
- Periodic DNS checks to ensure domain ownership hasn't changed
- Alert if DNS record is removed
- Grace period before disabling site
Verification Audit Log:
- Track when site was verified
- Record who performed verification
- Log IP address and timestamp
- DNS lookup results and timing

17 KiB Raw Blame History