Building Consent‑Aware Telemetry for Multi‑Tenant Rails: A Step‑by‑Step Guide

Black and white tiled interior with curved metal railings
A practical walkthrough shows how to add dynamic PII filtering to Rails logs, respecting user consent while keeping the data needed for debugging.

Introduction

In regulated industries, a single user’s consent revocation can turn a healthy Rails log into a compliance breach. Teams need a way to filter personally identifiable information (PII) dynamically, per‑tenant, without losing the context required for debugging. This guide walks through a complete project – from initial design to production launch – showing the decisions, milestones, and deliverables at each stage. By the end you’ll have a reusable pattern that integrates with Lescopr’s observability suite.

1. Project Kick‑off & Requirements Gathering

1.1 Define the compliance scope

  • Identify which data fields are considered PII under GDPR (email, phone, SSN, etc.).
  • Map each field to the Rails models that emit them (e.g., User, Order).
  • Determine consent granularity: global opt‑out vs. feature‑level consent.

1.2 Stakeholder alignment

Product: needs uninterrupted debugging information. Legal: requires zero PII leakage once consent is withdrawn. SRE: worries about added latency and cache consistency.

Deliverable: A requirements matrix that lists every PII attribute, its source, and the consent rule that applies.

2. Architecture Design

2.1 Choose the interception point

The most reliable place to filter logs is a Rack middleware that sits just before the Rails logger. This ensures every Rails.logger.info call passes through our filter.

2.2 Consent store integration

We use Lescopr’s Consent API as a single source of truth. The middleware queries the API (or a local cache) for the current tenant’s consent state before each log entry.

2.3 Redaction strategy

  • Pattern‑based redaction for known fields (e.g., regex for email).
  • Schema‑driven redaction for dynamic attributes using a JSON schema supplied by the product team.

Milestone: Architecture diagram approved by product and security leads.

3. Implementation Phase

3.1 Build the middleware

# app/middleware/consent_aware_logger.rb
class ConsentAwareLogger
  def initialize(app)
    @app = app
    @cache = ActiveSupport::Cache::MemoryStore.new(expires_in: 5.minutes)
  end

  def call(env)
    tenant_id = env["HTTP_X_TENANT_ID"]
    consent = fetch_consent(tenant_id)
    original_logger = Rails.logger
    Rails.logger = LoggerProxy.new(original_logger, consent)
    @app.call(env)
  ensure
    Rails.logger = original_logger
  end

  private

  def fetch_consent(tenant_id)
    @cache.fetch(tenant_id) { Lescopr::ConsentApi.state(tenant_id) }
  end
end

The LoggerProxy inspects each message, applies the redaction rules, and forwards the sanitized string.

3.2 Redaction engine

class LoggerProxy
  REDACT_PATTERNS = {
    email: /[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}/i,
    phone: /\b\d{3}[-.]?\d{3}[-.]?\d{4}\b/
  }

  def initialize(target, consent)
    @target = target
    @consent = consent
  end

  def info(message)
    @target.info(filter(message))
  end

  private

  def filter(msg)
    return msg unless @consent[:redact_pii]
    REDACT_PATTERNS.each_value { |r| msg = msg.gsub(r, '[REDACTED]') }
    msg
  end
end

3.3 Cache warm‑up job

A background job runs every minute to pre‑fetch consent states for active tenants, reducing cache‑miss latency.

Deliverable: Fully functional middleware package with test coverage > 90%.

4. Testing & Validation

4.1 Unit tests

  • Verify that messages containing an email are redacted when consent is false.
  • Ensure non‑PII logs remain untouched.

4.2 Integration tests

Simulate a request flow where a tenant toggles consent mid‑session. Assert that logs before the toggle contain PII and logs after the toggle are redacted.

4.3 Performance benchmarking

Measure request latency with and without the middleware using wrk. Target: < 5 ms overhead per request.

Result: Average overhead measured at 3.2 ms, well within the SRE budget.

5. Deployment Strategy

5.1 Canary release

  • Deploy the middleware to 5 % of production traffic.
  • Monitor log‑redaction rate, cache miss ratio, and error‑rate via Lescopr dashboards.

5.2 Roll‑out criteria

Metric Threshold
Redaction success ≥ 99.9 %
Cache miss ≤ 2 %
Added latency ≤ 5 ms

If all thresholds are met for 48 hours, increase rollout to 25 %, then 100 %.

6. Operational Monitoring & Incident Response

6.1 Dashboard widgets

  • PII Leak Indicator: counts raw PII strings detected post‑redaction.
  • Consent Cache Health: shows hit/miss ratio per tenant.
  • Latency Impact: aggregates request time before and after middleware activation.

6.2 Alerting rules

  • Trigger on any PII Leak Indicator > 0.
  • Alert if Cache miss exceeds 5 % for more than 10 minutes.

6.3 Run‑book excerpt

  1. Verify the offending tenant’s consent state via Lescopr UI.
  2. Flush the cache for that tenant.
  3. Redeploy the middleware if the issue persists.

7. Post‑Launch Review & Continuous Improvement

  • Conduct a privacy audit after 30 days to ensure no PII escaped.
  • Gather feedback from developers on log readability; adjust redaction patterns if debugging becomes too opaque.
  • Plan for schema‑driven redaction extensions to cover new data models.

Conclusion & Next Steps

Implementing consent‑aware telemetry in a multi‑tenant Rails environment requires careful coordination between logging, caching, and compliance layers. By following the milestones outlined above, teams can achieve GDPR‑compliant PII filtering while keeping the observability signal needed for rapid incident resolution. To go further, Lescopr's documentation covers step‑by‑step setup.