Episode 3 — NodeJS MongoDB Backend Architecture / 3.11 — Database Optimization

3.11 — Quick Revision: Database Optimization

Your cheat sheet for indexing, explain() analysis, aggregation pipeline stages, and MongoDB operators. Bookmark this page for quick reference during development and interview prep.


< Interview Questions | README >


Index Types

Index TypeSyntaxUse Case
Single-field{ email: 1 }Query/sort on one field
Compound{ status: 1, date: -1 }Multi-field queries (prefix rule)
Unique{ email: 1 }, { unique: true }Enforce uniqueness
Text{ name: "text", desc: "text" }Full-text search
TTL{ createdAt: 1 }, { expireAfterSeconds: 3600 }Auto-delete after time
Sparse{ phone: 1 }, { sparse: true }Skip docs missing the field
Partial{ price: 1 }, { partialFilterExpression: {...} }Index only matching docs
Wildcard{ "$**": 1 }Dynamic/flexible schemas

Index Commands

// Create
db.col.createIndex({ field: 1 });
db.col.createIndex({ a: 1, b: -1 }, { unique: true });

// List
db.col.getIndexes();

// Usage stats
db.col.aggregate([{ $indexStats: {} }]);

// Drop
db.col.dropIndex("field_1");
db.col.dropIndexes();              // Drop all except _id

// Mongoose
schema.index({ field: 1 });
await Model.createIndexes();

ESR Rule (Compound Index Field Order)

E — Equality    (exact match fields first)
S — Sort        (sort fields second)
R — Range       ($gte, $lte, $in fields last)
// Query: status = "active", sort by date, price >= 50
// Index: { status: 1, date: -1, price: 1 }
//           E            S          R

explain() Output Key Fields

FieldWhat It Tells YouGood Value
stageScan typeIXSCAN (not COLLSCAN)
nReturnedDocs returnedExpected count
totalKeysExaminedIndex entries scannedClose to nReturned
totalDocsExaminedDocs read from diskClose to nReturned (0 for covered)
executionTimeMillisQuery timeAs low as possible
db.col.find({...}).explain("executionStats");

Aggregation Pipeline Stages

StagePurposeExample
$matchFilter documents{ $match: { status: "active" } }
$groupGroup + accumulate{ $group: { _id: "$field", total: { $sum: "$amt" } } }
$projectReshape / select fields{ $project: { name: 1, total: 1, _id: 0 } }
$addFieldsAdd fields (keep existing){ $addFields: { tax: { $multiply: ["$total", 0.18] } } }
$sortOrder results{ $sort: { total: -1 } }
$limitLimit count{ $limit: 10 }
$skipSkip N docs{ $skip: 20 }
$unwindFlatten arrays{ $unwind: "$tags" }
$lookupJoin collections{ $lookup: { from, localField, foreignField, as } }
$countCount docs{ $count: "total" }
$facetParallel pipelines{ $facet: { results: [...], count: [...] } }
$bucketManual bucketing{ $bucket: { groupBy, boundaries, default, output } }
$bucketAutoAuto bucketing{ $bucketAuto: { groupBy, buckets } }
$graphLookupRecursive traversal{ $graphLookup: { from, startWith, connectFromField, connectToField, as } }
$mergeUpsert to collection{ $merge: { into, on, whenMatched, whenNotMatched } }
$outReplace collection{ $out: "collectionName" }

Accumulators ($group)

AccumulatorPurposeExample
$sumSum / count{ $sum: "$amount" } or { $sum: 1 }
$avgAverage{ $avg: "$price" }
$minMinimum{ $min: "$date" }
$maxMaximum{ $max: "$score" }
$pushCollect into array{ $push: "$name" }
$addToSetCollect unique values{ $addToSet: "$tag" }
$firstFirst in group{ $first: "$name" }
$lastLast in group{ $last: "$name" }
$countCount in group{ $count: {} }

Comparison Operators

OperatorMeaningExample
$eqEqual{ age: { $eq: 25 } }
$neNot equal{ status: { $ne: "deleted" } }
$gtGreater than{ price: { $gt: 50 } }
$gteGreater or equal{ age: { $gte: 18 } }
$ltLess than{ stock: { $lt: 10 } }
$lteLess or equal{ rating: { $lte: 3 } }
$inIn array{ status: { $in: ["a", "b"] } }
$ninNot in array{ role: { $nin: ["banned"] } }

Logical Operators

OperatorSyntax
$and{ $and: [ {cond1}, {cond2} ] }
$or{ $or: [ {cond1}, {cond2} ] }
$not{ field: { $not: { $gt: 100 } } }
$nor{ $nor: [ {cond1}, {cond2} ] }

Note: Implicit $and is the default when specifying multiple conditions on the same query.


Array Operators

Query

OperatorPurposeExample
$elemMatchElement matches all conditions{ scores: { $elemMatch: { $gte: 80, $lte: 90 } } }
$allArray has all values{ tags: { $all: ["a", "b"] } }
$sizeExact array length{ items: { $size: 3 } }

Update

OperatorPurposeExample
$pushAdd element{ $push: { tags: "new" } }
$addToSetAdd if unique{ $addToSet: { tags: "new" } }
$pullRemove matching{ $pull: { tags: "old" } }
$popRemove first/last{ $pop: { tags: 1 } } (1=last, -1=first)

Update Operators

OperatorPurposeExample
$setSet field value{ $set: { name: "Alice" } }
$unsetRemove field{ $unset: { temp: "" } }
$incIncrement{ $inc: { count: 1 } }
$mulMultiply{ $mul: { price: 0.9 } }
$renameRename field{ $rename: { "old": "new" } }
$minSet if less{ $min: { low: 50 } }
$maxSet if greater{ $max: { high: 99 } }
$currentDateSet current date{ $currentDate: { updatedAt: true } }

Element and Regex Operators

OperatorPurposeExample
$existsField exists{ phone: { $exists: true } }
$typeField BSON type{ age: { $type: "number" } }
$regexPattern match{ name: { $regex: /^alice/i } }

Performance Checklist

  • $match is the first stage in aggregation pipelines
  • Compound indexes follow ESR order
  • explain("executionStats") shows IXSCAN, not COLLSCAN
  • totalDocsExamined is close to nReturned
  • No unused indexes wasting RAM (check $indexStats)
  • Index total size fits in available RAM
  • allowDiskUse: true set for large aggregations
  • Auto-indexing disabled in production (autoIndex: false)
  • TTL indexes used for session/token cleanup
  • $lookup has indexes on foreignField