Introduction

Compliance and observability often feel like opposing goals: you want deep insight into every request, yet GDPR forces you to limit personal data collection. This guide walks you through a full project—from initial design to production launch—showing the exact milestones, decisions, and deliverables needed to build a GDPR‑compliant observability pipeline for a Node.js service. By following each step you will retain the richness of tracing, metrics, and logs while respecting user consent.

1. Project Planning & Requirements

1.1 Define Compliance Scope

What personal data does your service handle? Identify any PII (email, IP address, user‑agent) that could be captured by tracing or logging libraries.

List data categories
Map each category to GDPR legal basis (consent, contract, legitimate interest)
Decide which categories must be masked or omitted from observability streams

1.2 Set Observability Goals

Latency visibility – target < 200 ms 99th‑percentile for critical endpoints
Error rate – aim for MTTR < 30 minutes after an incident
Compliance metrics – ensure 100 % of traces respect consent flags

1.3 Choose the Stack

Layer	Recommended Tool
Tracing	OpenTelemetry SDK for Node.js
Metrics	Prometheus client for Node
Logs	Winston with GDPR‑aware formatter
Consent Management	Lescopr Consent SDK
Dashboard	Lescopr Observability UI

2. Architecture & Data Flow Design

2.1 High‑Level Diagram

Client → API Gateway → Node.js Service → OpenTelemetry → Collector → Lescopr Backend → Dashboard

The API Gateway extracts consent headers (X‑User‑Consent) and forwards them as context.
The Node.js Service injects the consent flag into the OpenTelemetry span attributes.
The Collector filters spans that lack explicit consent before forwarding to storage.

2.2 Consent Propagation Strategy

Middleware – add a lightweight Express middleware that reads the consent header and stores it in the request context.
Span Enrichment – use OpenTelemetry’s setAttribute to attach user.consent: true/false.
Exporter Filtering – configure the OTLP exporter to drop spans where user.consent is false.

3. Implementation Phase

3.1 Scaffold the Project

mkdir gdpr‑observability && cd gdpr‑observability
npm init -y
npm install express @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/exporter-otlp-grpc lescopr-consent winston

3.2 Add Consent Middleware (H3)

// consentMiddleware.js
module.exports = function consentMiddleware(req, res, next) {
  const consentHeader = req.headers['x-user-consent'];
  req.consent = consentHeader === 'true';
  next();
};

3.3 Initialise OpenTelemetry with Consent Filtering

// tracing.js
const { NodeTracerProvider } = require('@opentelemetry/sdk-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-otlp-grpc');
const { SimpleSpanProcessor } = require('@opentelemetry/sdk-trace-base');

const provider = new NodeTracerProvider();
const exporter = new OTLPTraceExporter({
  url: 'grpc://localhost:4317',
});

provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register();

// Filter spans without consent
provider.addSpanProcessor({
  onStart(span, ctx) {
    const consent = ctx.getValue('user.consent');
    if (!consent) {
      span.setAttribute('user.consent', false);
    }
  },
  onEnd(span) {}
});

3.4 Integrate Middleware & Tracing in Express

const express = require('express');
const consent = require('./consentMiddleware');
require('./tracing'); // side‑effect init

const app = express();
app.use(consent);

app.get('/api/data', (req, res) => {
  // business logic
  res.json({ ok: true });
});

app.listen(3000, () => console.log('Service listening on port 3000'));

3.5 Logging with GDPR‑Aware Formatter

const winston = require('winston');
const { LescoprConsentFormat } = require('lescopr-consent');

const logger = winston.createLogger({
  level: 'info',
  format: LescoprConsentFormat({ maskFields: ['email', 'ip'] }),
  transports: [new winston.transports.Console()],
});

4. Testing & Validation

4.1 Unit Tests for Consent Propagation

Mock requests with X‑User‑Consent: true and verify req.consent is true.
Ensure spans contain user.consent attribute.

4.2 End‑to‑End Load Test

Use k6 to simulate 1 000 RPS, toggling consent header randomly. Verify that:

Latency stays below the 200 ms target.
No span without consent reaches the collector (inspect collector logs).

4.3 GDPR Auditing Checklist

All PII fields are masked in logs.
No trace contains personal identifiers when consent is false.
Data retention policies in Lescopr are set to 30 days for consented data.

5. Deployment & Monitoring

5.1 CI/CD Pipeline

Lint & Unit Test – npm run lint && npm test
Docker Build – multi‑stage Dockerfile, copy only compiled code.
Canary Release – deploy to a staging namespace, route 5 % of traffic.
Observability Validation – automatic health‑check that queries Lescopr dashboard for consent compliance metrics.

5.2 Production Roll‑out

Gradually increase traffic to 100 % once compliance dashboards show 0 % non‑consented spans.
Set up alert: If consent‑filtered span rate > 0.1 %, trigger PagerDuty incident.

6. Ongoing Governance

Quarterly Review – audit consent flag handling against updated GDPR guidance.
Performance Tuning – adjust sampling rates in OpenTelemetry to keep exporter bandwidth under 2 Mbps.
Documentation – keep internal runbooks aligned with Lescopr’s public docs.

Conclusion

By following this checklist you have built a Node.js observability pipeline that respects GDPR consent, maintains low latency, and provides actionable metrics for SRE teams. The project demonstrates that privacy and performance can coexist when you embed consent awareness at every layer of the data path.

To go further, Lescopr's documentation covers step‑by‑step setup.