3.12.c — Error Handling and Logging

When errors happen in production, two things determine how fast you fix them: the quality of your error logs and the tools you use to find them. Centralized error logging with request correlation, stack traces, and external monitoring turns reactive firefighting into proactive debugging.

---

< 3.12.b — Logging Libraries | Exercise Questions >

---

1. Logging Stack Traces

A stack trace tells you exactly where an error occurred. Always capture and log it:

const logger = require("./config/logger");

// BAD — loses the stack trace
try {
  await processOrder(orderId);
} catch (err) {
  logger.error("Order failed: " + err.message);
  // Missing: WHERE in the code did it fail?
}

// GOOD — includes full stack trace
try {
  await processOrder(orderId);
} catch (err) {
  logger.error("Order processing failed", {
    error: err.message,
    stack: err.stack,
    orderId,
    userId: req.user._id,
  });
}

Winston Error Format

Winston can automatically include stack traces with the errors format:

const winston = require("winston");

const logger = winston.createLogger({
  format: winston.format.combine(
    winston.format.timestamp(),
    winston.format.errors({ stack: true }),  // Captures stack traces
    winston.format.json()
  ),
  transports: [
    new winston.transports.File({ filename: "logs/error.log", level: "error" }),
  ],
});

// Now you can pass Error objects directly
logger.error(new Error("Database connection failed"));
// Output includes full stack:
// {
//   "level": "error",
//   "message": "Database connection failed",
//   "stack": "Error: Database connection failed\n    at Object.<anonymous> (/app/db.js:15:9)\n    ...",
//   "timestamp": "2025-06-15T14:23:45.123Z"
// }

---

2. Centralized Error Logger

Create a centralized error handling utility instead of scattering try/catch blocks everywhere:

// utils/errorLogger.js
const logger = require("../config/logger");

class AppError extends Error {
  constructor(message, statusCode, isOperational = true) {
    super(message);
    this.statusCode = statusCode;
    this.isOperational = isOperational;  // Expected errors (vs programming bugs)
    Error.captureStackTrace(this, this.constructor);
  }
}

const logError = (err, context = {}) => {
  const logData = {
    error: err.message,
    stack: err.stack,
    statusCode: err.statusCode || 500,
    isOperational: err.isOperational || false,
    ...context,
  };

  if (err.isOperational) {
    logger.warn("Operational error", logData);
  } else {
    logger.error("Unexpected error", logData);
  }
};

module.exports = { AppError, logError };

Using the Centralized Error Logger

const { AppError, logError } = require("./utils/errorLogger");

// In route handlers
app.post("/api/orders", async (req, res, next) => {
  try {
    const order = await createOrder(req.body);
    res.status(201).json(order);
  } catch (err) {
    logError(err, {
      requestId: req.id,
      userId: req.user?._id,
      body: req.body,
    });
    next(err);
  }
});

// Throwing operational errors
const getUser = async (id) => {
  const user = await User.findById(id);
  if (!user) {
    throw new AppError("User not found", 404);
  }
  return user;
};

---

3. Express Error-Handling Middleware

A global error handler catches all errors in the Express pipeline:

// middleware/errorHandler.js
const logger = require("../config/logger");

const errorHandler = (err, req, res, next) => {
  // Log the error with full context
  logger.error("Request error", {
    error: err.message,
    stack: err.stack,
    statusCode: err.statusCode || 500,
    method: req.method,
    url: req.originalUrl,
    requestId: req.id,
    userId: req.user?._id,
    ip: req.ip,
    userAgent: req.get("User-Agent"),
  });

  // Determine status code
  const statusCode = err.statusCode || 500;

  // Send response
  res.status(statusCode).json({
    status: "error",
    message: statusCode === 500 ? "Internal server error" : err.message,
    ...(process.env.NODE_ENV === "development" && {
      stack: err.stack,
      details: err.details,
    }),
  });
};

module.exports = errorHandler;

// app.js — register AFTER all routes
app.use("/api/users", userRoutes);
app.use("/api/orders", orderRoutes);

// 404 handler
app.use((req, res) => {
  logger.warn("Route not found", { method: req.method, url: req.originalUrl });
  res.status(404).json({ status: "error", message: "Route not found" });
});

// Global error handler (must have 4 parameters)
app.use(errorHandler);

---

4. Unhandled Rejections and Uncaught Exceptions

These are the last line of defense — catch errors that escape all other handlers:

const logger = require("./config/logger");

// Unhandled Promise Rejections
process.on("unhandledRejection", (reason, promise) => {
  logger.error("Unhandled Promise Rejection", {
    reason: reason instanceof Error ? reason.message : reason,
    stack: reason instanceof Error ? reason.stack : undefined,
  });
  // In production, you may want to exit gracefully:
  // process.exit(1);
});

// Uncaught Exceptions
process.on("uncaughtException", (err) => {
  logger.error("Uncaught Exception — shutting down", {
    error: err.message,
    stack: err.stack,
  });
  // MUST exit — the process is in an undefined state
  process.exit(1);
});

// SIGTERM (graceful shutdown)
process.on("SIGTERM", () => {
  logger.info("SIGTERM received — shutting down gracefully");
  server.close(() => {
    logger.info("Server closed");
    process.exit(0);
  });
});

Important: After an uncaughtException, the process state is unreliable. Always exit and let a process manager (PM2, Docker, systemd) restart the application.

---

5. Request ID Correlation

When a single user action generates multiple log entries across services, a request ID ties them all together:

// middleware/requestId.js
const { v4: uuidv4 } = require("uuid");

const requestIdMiddleware = (req, res, next) => {
  // Use existing header (from API gateway) or generate new
  req.id = req.headers["x-request-id"] || uuidv4();
  res.setHeader("x-request-id", req.id);
  next();
};

module.exports = requestIdMiddleware;

// app.js — add BEFORE other middleware
app.use(requestIdMiddleware);
app.use(morgan("combined", { stream: morganStream }));

Using Request ID in Logs

// Every log in the request lifecycle includes the requestId
app.post("/api/orders", async (req, res, next) => {
  const { id: requestId } = req;
  const userId = req.user._id;

  logger.info("Order request received", { requestId, userId });

  try {
    logger.info("Validating order data", { requestId, userId });
    const validated = validateOrder(req.body);

    logger.info("Processing payment", { requestId, userId, amount: validated.total });
    const payment = await processPayment(validated);

    logger.info("Creating order record", { requestId, userId, paymentId: payment.id });
    const order = await Order.create({ ...validated, paymentId: payment.id });

    logger.info("Order created successfully", { requestId, userId, orderId: order._id });
    res.status(201).json(order);
  } catch (err) {
    logger.error("Order creation failed", { requestId, userId, error: err.message, stack: err.stack });
    next(err);
  }
});

Now you can trace a complete request by searching for the requestId:

Search logs for requestId: "req-abc-123"
┌─────────────────────────────────────────────────────────────────┐
│ 14:23:45.100 [INFO]  Order request received     {req-abc-123}  │
│ 14:23:45.102 [INFO]  Validating order data      {req-abc-123}  │
│ 14:23:45.110 [INFO]  Processing payment         {req-abc-123}  │
│ 14:23:45.450 [ERROR] Order creation failed       {req-abc-123}  │
│                        → Payment gateway timeout                 │
└─────────────────────────────────────────────────────────────────┘

---

6. Async Local Storage (Advanced Request Context)

Node.js AsyncLocalStorage allows automatic context propagation without passing requestId everywhere:

// config/asyncContext.js
const { AsyncLocalStorage } = require("async_hooks");
const asyncLocalStorage = new AsyncLocalStorage();

module.exports = asyncLocalStorage;

// middleware/contextMiddleware.js
const asyncLocalStorage = require("../config/asyncContext");
const { v4: uuidv4 } = require("uuid");

const contextMiddleware = (req, res, next) => {
  const requestId = req.headers["x-request-id"] || uuidv4();
  const store = { requestId, userId: req.user?._id };

  asyncLocalStorage.run(store, () => {
    req.id = requestId;
    res.setHeader("x-request-id", requestId);
    next();
  });
};

// config/logger.js — automatically include context
const asyncLocalStorage = require("./asyncContext");

const contextFormat = winston.format((info) => {
  const store = asyncLocalStorage.getStore();
  if (store) {
    info.requestId = store.requestId;
    info.userId = store.userId;
  }
  return info;
});

// Add to format chain:
format: winston.format.combine(
  contextFormat(),
  winston.format.timestamp(),
  winston.format.json()
)

// Now EVERY log automatically includes requestId and userId
// without explicitly passing them:
logger.info("Payment processed");
// {"level":"info","message":"Payment processed","requestId":"req-abc-123","userId":"user-456","timestamp":"..."}

---

7. Sentry — Error Monitoring

Sentry is a cloud-based error monitoring platform that captures errors, stack traces, and context automatically:

npm install @sentry/node

const Sentry = require("@sentry/node");

Sentry.init({
  dsn: process.env.SENTRY_DSN,
  environment: process.env.NODE_ENV,
  tracesSampleRate: 0.1,    // Sample 10% of transactions for performance
});

// Express integration
app.use(Sentry.Handlers.requestHandler());
app.use(Sentry.Handlers.tracingHandler());

// ... routes ...

// Error handler — BEFORE your custom error handler
app.use(Sentry.Handlers.errorHandler());
app.use(errorHandler);  // Your custom handler

What Sentry Provides

Sentry Dashboard:
┌───────────────────────────────────────────────────────┐
│ Error: MongoNetworkError — connect ECONNREFUSED        │
│ First seen: 2025-06-15 14:23:45                       │
│ Last seen: 2025-06-15 14:25:12                        │
│ Events: 47                                            │
│ Users affected: 23                                    │
│ Stack trace: db.js:15 → orderService.js:42 → route.js │
│ Tags: env=production, service=order-api                │
│ Breadcrumbs: HTTP GET /api/orders → DB query → ERROR   │
│ [Resolve] [Ignore] [Assign to Team]                   │
└───────────────────────────────────────────────────────┘

---

8. LogRocket — Session Replay

LogRocket captures user sessions including network requests, console logs, and UI interactions. It helps you see exactly what the user experienced when an error occurred.

// Frontend integration (React example)
import LogRocket from "logrocket";
LogRocket.init("your-app-id/project");

// Identify user
LogRocket.identify(userId, {
  name: user.name,
  email: user.email,
});

Key difference from Sentry:

• Sentry captures errors and stack traces (backend-focused)
• LogRocket captures user sessions and UI replay (frontend-focused)
• They complement each other — use both for full visibility

---

9. ELK Stack — Log Aggregation (Concept)

The ELK stack is the industry standard for centralized log management:

ELK Stack Architecture:
┌────────────┐    ┌────────────────┐    ┌─────────────┐    ┌──────────┐
│  Your App  │───→│  Logstash /    │───→│ Elasticsearch│───→│  Kibana  │
│  (Winston) │    │  Filebeat      │    │  (Storage +  │    │  (UI +   │
│            │    │  (Ingestion)   │    │   Search)    │    │  Graphs) │
└────────────┘    └────────────────┘    └─────────────┘    └──────────┘

Component	Role	Analogy
Elasticsearch	Store and search logs	The database for logs
Logstash	Ingest, transform, forward logs	The pipeline
Kibana	Visualize and search logs	The dashboard
Filebeat	Lightweight log shipper	The collector (alternative to Logstash)

How It Works

1. Your app writes structured JSON logs (via Winston/Pino)
2. Filebeat reads log files and sends them to Elasticsearch
3. Elasticsearch indexes the logs for fast searching
4. Kibana provides a web UI to search, filter, and visualize logs

Cloud Alternatives

Service	Provider	Description
CloudWatch Logs	AWS	Built-in AWS log aggregation
Stackdriver / Cloud Logging	GCP	Google Cloud logging
Datadog	Third-party	Monitoring + logging platform
Logtail (Better Stack)	Third-party	Modern log management
Papertrail	Third-party	Simple cloud log aggregation

---

10. Complete Error Handling + Logging Flow

Request lifecycle with logging:
┌──────────────────────────────────────────────────────────────────┐
│                                                                  │
│  Request arrives                                                 │
│  ├── [Request ID Middleware] → assign/extract requestId          │
│  ├── [Morgan] → log HTTP method, URL, start time                 │
│  ├── [Auth Middleware] → log auth success/failure                 │
│  ├── [Route Handler]                                             │
│  │   ├── logger.info("Processing...", { requestId, userId })     │
│  │   ├── try { ... business logic ... }                          │
│  │   └── catch (err) → logError(err, { requestId, userId })     │
│  ├── [Error Handler Middleware] → log error, send response       │
│  └── [Morgan] → log HTTP status, response time                   │
│                                                                  │
│  Background:                                                     │
│  ├── [unhandledRejection handler] → log + alert                  │
│  ├── [uncaughtException handler] → log + exit                    │
│  └── [Sentry] → capture + notify team                            │
│                                                                  │
│  Storage:                                                        │
│  ├── logs/combined-2025-06-15.log (all logs)                     │
│  ├── logs/error-2025-06-15.log (errors only)                     │
│  └── Sentry/ELK/CloudWatch (external monitoring)                 │
│                                                                  │
└──────────────────────────────────────────────────────────────────┘

---

Key Takeaways

1. Always log stack traces — use Winston's errors({ stack: true }) format
2. Centralize error handling — one AppError class, one logError utility, one error middleware
3. Catch unhandled rejections and uncaught exceptions — they are your last line of defense
4. Use request IDs to correlate all logs from a single request across services
5. AsyncLocalStorage enables automatic context propagation without manual parameter passing
6. Sentry captures errors in real-time with stack traces, affected users, and alerting
7. ELK stack (Elasticsearch + Logstash + Kibana) is the industry standard for log aggregation
8. Operational errors vs programming errors — handle them differently (respond vs crash)

---

Explain-It Challenge

A user reports: "I clicked 'Place Order' and nothing happened." This happened 2 hours ago. Design the complete investigation workflow: which logs would you search first, how would you use the request ID to trace the request, what tools (Sentry, ELK, CloudWatch) would you check, and what information in the logs would help you identify the root cause? Also explain how you would prevent this class of error from going unnoticed in the future.