Table of Contents

1. Introduction to Indexes in MongoDB
2. Why Indexes Matter
3. Types of Indexes in MongoDB
   • Single Field Indexes
   • Compound Indexes
   • Multikey Indexes
   • Text Indexes
   • Geospatial Indexes
   • Hashed Indexes
4. Creating Indexes in MongoDB
5. Viewing Existing Indexes
6. Index Use Case Scenarios
7. Indexes and Performance
8. Indexing Best Practices
9. When Not to Use Indexes
10. Conclusion

---

1. Introduction to Indexes in MongoDB

Indexes are a crucial aspect of MongoDB that significantly enhance query performance. By default, MongoDB performs a collection scan, which means it examines every document in the collection to match the query. Indexes act like shortcuts that help MongoDB locate the relevant documents much faster, akin to the index of a book.

Without proper indexing, even the most powerful servers can experience sluggish query performance as the data volume grows. Therefore, understanding and using indexes correctly is fundamental when building scalable MongoDB applications.

---

2. Why Indexes Matter

Imagine running a query on a collection of millions of documents. Without an index, MongoDB has to scan through each document to find matches, which is computationally expensive. Indexes provide an efficient way to locate and retrieve only those documents that match the query criteria.

Key benefits of indexing include:

• Faster data retrieval
• Improved query efficiency
• Better scalability for large datasets
• Support for unique constraints and data validation

---

3. Types of Indexes in MongoDB

MongoDB supports a wide variety of index types tailored for different use cases. Here’s a breakdown of the most important ones:

Single Field Indexes

The most basic type. Created on a single field, it supports quick lookup and sorting.

db.users.createIndex({ "email": 1 })

Compound Indexes

Indexes on multiple fields, often used to support more complex queries.

db.orders.createIndex({ "userId": 1, "createdAt": -1 })

The order of fields in a compound index matters and affects its usability for different queries.

Multikey Indexes

Automatically created when you index a field that contains an array. MongoDB indexes each element of the array.

db.products.createIndex({ "tags": 1 })

Text Indexes

Used for full-text search in string content fields.

db.articles.createIndex({ content: "text" })

Text indexes support searching via $text queries.

Geospatial Indexes

Ideal for location-based queries like finding nearby places. Two types are supported:

• 2d
• 2dsphere

db.places.createIndex({ location: "2dsphere" })

Hashed Indexes

Used for sharding and distributing data evenly across shards. Not optimal for range queries.

db.customers.createIndex({ userId: "hashed" })

---

4. Creating Indexes in MongoDB

Indexes can be created with createIndex() or ensureIndex() (deprecated). Here’s a basic example:

db.customers.createIndex({ "email": 1 })

Options include:

• unique: Ensures values in the indexed field are unique.
• background: Builds the index in the background without locking writes.
• sparse: Only indexes documents where the field exists.
• expireAfterSeconds: Used for TTL indexes.

Example with options:

db.sessions.createIndex({ "lastAccessed": 1 }, { expireAfterSeconds: 3600 })

---

5. Viewing Existing Indexes

To list all indexes on a collection:

db.customers.getIndexes()

You can also drop an index:

db.customers.dropIndex("email_1")

Or drop all indexes:

db.customers.dropIndexes()

---

6. Index Use Case Scenarios

Use Case 1: Unique Email Addresses

db.users.createIndex({ email: 1 }, { unique: true })

Ensures no duplicate email IDs are stored.

Use Case 2: Blog Search

db.blogs.createIndex({ content: "text", title: "text" })

Enables full-text search across articles.

Use Case 3: Location-Based Services

db.stores.createIndex({ location: "2dsphere" })

Find nearest stores within a certain radius.

---

7. Indexes and Performance

You can use explain() to understand how MongoDB is using indexes:

db.users.find({ email: "[email protected]" }).explain("executionStats")

Look for IXSCAN instead of COLLSCAN to ensure indexes are being used.

Be mindful of the index size, as it grows with data. Too many indexes can degrade write performance and increase memory usage.

---

8. Indexing Best Practices

• Index fields used in queries: Especially for frequent filters, sorts, and joins.
• Avoid over-indexing: Indexes improve reads but slow down writes (due to index maintenance).
• Use compound indexes wisely: Combine fields often queried together.
• Use hint() to force index usage when necessary.
• Monitor slow queries: Use MongoDB Atlas profiler or logs to detect unindexed queries.

---

9. When Not to Use Indexes

There are scenarios where indexes can backfire:

• Fields with high write frequency and low query use
• Low cardinality fields (e.g., gender: male/female)
• Very small datasets where collection scans are faster

Every index consumes disk space and adds overhead to insert/update operations, so be judicious in index design.

---

10. Conclusion

Indexes in MongoDB are indispensable tools for optimizing read operations and enabling scalable, performant applications. With several index types like compound, multikey, text, and geospatial, MongoDB provides a flexible indexing system to suit various needs.

Understanding when and how to use indexes, along with regular profiling of queries, is key to keeping your application fast and efficient as it grows in complexity and size.

---

Table of Contents

1. Introduction
2. Why Use Aggregation for Analytics?
3. Common Stages in Analytics Pipelines
4. Designing a Real-World Analytics Pipeline
5. Step-by-Step Example: E-commerce Sales Dashboard
6. Best Practices for Analytics Aggregations
7. Performance Tips
8. Conclusion

---

Introduction

In modern applications, analytics and reporting are crucial for understanding user behavior, product performance, and business trends. MongoDB’s Aggregation Framework is powerful enough to perform real-time data analytics, even over large collections, without exporting data to external systems.

---

Why Use Aggregation for Analytics?

MongoDB’s aggregation pipeline allows you to:

• Group, sort, and filter large datasets efficiently.
• Perform calculations like averages, totals, and percentages.
• Join data from other collections.
• Shape and transform your output for dashboards or APIs.
• Embed complex logic into a single query.

This enables MongoDB to act as both a transactional and analytical database for many applications.

---

Common Stages in Analytics Pipelines

Here are the most frequently used aggregation stages in analytics use cases:

Stage	Purpose
$match	Filter documents for specific time periods or users
$group	Summarize data by category, date, user, etc.
$project	Reshape documents, compute derived fields
$sort	Sort analytics results (e.g., top 10 products)
$count	Count the number of documents in a subset
$bucket	Group by value ranges (age groups, price ranges)
$facet	Run multiple aggregations in parallel
$lookup	Join data across collections
$filter	Filter array fields before further aggregation

---

Designing a Real-World Analytics Pipeline

Suppose you’re building a sales dashboard. Some key analytics requirements might be:

• Daily sales totals
• Most sold products
• Average order value
• User purchase frequency
• Time-based trends

To support this, you need an aggregation pipeline that processes data efficiently from your orders collection.

---

Step-by-Step Example: E-commerce Sales Dashboard

Collection: orders

{
  "_id": ObjectId("..."),
  "userId": ObjectId("..."),
  "items": [
    { "productId": "p1", "quantity": 2, "price": 150 },
    { "productId": "p2", "quantity": 1, "price": 200 }
  ],
  "total": 500,
  "createdAt": ISODate("2024-03-01T12:00:00Z")
}

---

Example: Get Daily Sales Summary

db.orders.aggregate([
  {
    $match: {
      createdAt: {
        $gte: ISODate("2024-03-01T00:00:00Z"),
        $lt: ISODate("2024-04-01T00:00:00Z")
      }
    }
  },
  {
    $group: {
      _id: { $dateToString: { format: "%Y-%m-%d", date: "$createdAt" } },
      totalRevenue: { $sum: "$total" },
      orderCount: { $sum: 1 }
    }
  },
  {
    $project: {
      date: "$_id",
      totalRevenue: 1,
      orderCount: 1,
      avgOrderValue: { $divide: ["$totalRevenue", "$orderCount"] }
    }
  },
  { $sort: { date: 1 } }
])

---

Result:

[
  {
    "date": "2024-03-01",
    "totalRevenue": 12000,
    "orderCount": 40,
    "avgOrderValue": 300
  },
  ...
]

This pipeline:

• Filters data for March 2024
• Groups orders by date
• Calculates total revenue, order count, and average value
• Sorts the results chronologically

---

Additional Example: Top 5 Most Sold Products

db.orders.aggregate([
  { $unwind: "$items" },
  {
    $group: {
      _id: "$items.productId",
      totalSold: { $sum: "$items.quantity" },
      revenue: { $sum: { $multiply: ["$items.quantity", "$items.price"] } }
    }
  },
  { $sort: { totalSold: -1 } },
  { $limit: 5 }
])

---

Best Practices for Analytics Aggregations

• Use $project early to reduce document size.
• Use $match to filter data early and reduce processing load.
• Use indexes to optimize $match and $sort.
• Structure documents to reduce the need for $lookup if possible.
• Cache results for heavy aggregation queries when appropriate.

---

Performance Tips

Strategy	Benefit
Use compound indexes	Boost $match + $sort performance
Avoid unnecessary $lookup	Reduce latency
Use $merge or $out	Store and reuse analytics results
Batch time-consuming pipelines	Schedule as background tasks
Use Atlas Triggers or Change Streams	Generate real-time analytics

---

Conclusion

MongoDB’s Aggregation Framework allows you to build powerful, expressive analytics pipelines directly inside the database. With the right design and performance optimizations, you can deliver fast, real-time insights without additional ETL layers.

---

Updated Table of Contents

1. Introduction to Advanced Aggregation
2. $unwind – Deconstructing Arrays
3. $lookup – Performing Joins in MongoDB
4. $facet – Multi-Faceted Aggregation
5. $bucket – Grouping Data into Ranges
6. $filter – Filtering Arrays in Aggregation
7. Real-World Example Combining These Stages
8. Conclusion

---

Introduction to Advanced Aggregation

MongoDB’s aggregation pipeline becomes incredibly powerful when you go beyond the basics. This module covers four advanced stages that are crucial for performing complex data operations:

• $unwind: Flattens arrays.
• $lookup: Performs left outer joins.
• $facet: Allows parallel pipelines for diverse analysis.
• $bucket: Groups data by value ranges.

Let’s explore each with examples.

---

$unwind – Deconstructing Arrays

The $unwind stage breaks an array field into multiple documents, one for each element.

Syntax:

{ $unwind: "$arrayField" }

Example:

db.orders.aggregate([
  { $unwind: "$items" }
])

If an order document has an items array:

{
  orderId: 1,
  items: ["pen", "notebook", "eraser"]
}

After $unwind, it becomes:

{ orderId: 1, items: "pen" }
{ orderId: 1, items: "notebook" }
{ orderId: 1, items: "eraser" }

✅ Use Case: Required when calculating statistics per array item (e.g., each item sold).

---

$lookup – Performing Joins in MongoDB

The $lookup stage is MongoDB’s way to perform SQL-style joins.

Syntax:

{
  $lookup: {
    from: "collectionToJoin",
    localField: "fieldInCurrentCollection",
    foreignField: "fieldInOtherCollection",
    as: "joinedData"
  }
}

Example:

db.orders.aggregate([
  {
    $lookup: {
      from: "customers",
      localField: "customerId",
      foreignField: "_id",
      as: "customerDetails"
    }
  }
])

Each order will now include customer info in an array field customerDetails.

✅ Pro Tip: Use $unwind on the joined field to flatten it into a single object if each order is tied to one customer.

---

$facet – Multi-Faceted Aggregation

The $facet stage allows you to run multiple aggregation pipelines in parallel and return results in one document. This is useful for analytics dashboards.

Syntax:

{
  $facet: {
    pipeline1Name: [ /* stages */ ],
    pipeline2Name: [ /* stages */ ],
    ...
  }
}

Example:

db.products.aggregate([
  {
    $facet: {
      priceStats: [
        { $group: { _id: null, avgPrice: { $avg: "$price" }, maxPrice: { $max: "$price" } } }
      ],
      categoryCount: [
        { $group: { _id: "$category", count: { $sum: 1 } } }
      ]
    }
  }
])

Returns a single document like:

{
  "priceStats": [{ "avgPrice": 35.5, "maxPrice": 100 }],
  "categoryCount": [{ "_id": "Books", "count": 10 }, ...]
}

✅ Use Case: Ideal for analytics, reporting, or dashboard queries.

---

$bucket – Grouping Data into Ranges

The $bucket stage groups documents based on specified boundaries of a field (like age or price).

Syntax:

{
  $bucket: {
    groupBy: "$price",
    boundaries: [0, 50, 100, 150],
    default: "Others",
    output: {
      count: { $sum: 1 },
      products: { $push: "$name" }
    }
  }
}

Example:

db.products.aggregate([
  {
    $bucket: {
      groupBy: "$price",
      boundaries: [0, 50, 100],
      default: "Expensive",
      output: {
        count: { $sum: 1 },
        items: { $push: "$name" }
      }
    }
  }
])

This groups products into price ranges: 0–49, 50–99, and the rest as "Expensive".

✅ Use Case: Great for histogram-style data like age brackets, price ranges, etc.

$filter – Filtering Arrays in Aggregation

The $filter operator allows you to return only specific elements from an array that match a certain condition. It’s extremely useful when you don’t want to unwind the array, but only want relevant values retained.

Syntax:

{
  $filter: {
    input: "<arrayField>",
    as: "<variableName>",
    cond: { <condition expression using the variable> }
  }
}

Example: Filter completed tasks only

Assume we have a tasks array inside user documents:

{
  name: "Alice",
  tasks: [
    { title: "Task 1", completed: true },
    { title: "Task 2", completed: false },
    { title: "Task 3", completed: true }
  ]
}

We can filter only the completed tasks using:

db.users.aggregate([
  {
    $project: {
      name: 1,
      completedTasks: {
        $filter: {
          input: "$tasks",
          as: "task",
          cond: { $eq: ["$$task.completed", true] }
        }
      }
    }
  }
])

Result:

{
  name: "Alice",
  completedTasks: [
    { title: "Task 1", completed: true },
    { title: "Task 3", completed: true }
  ]
}

---

✅ Use Cases of $filter:

• Show only high-rated products from an embedded reviews array
• Return users who have only completed certain badges or certifications
• Simplify array-based filtering without flattening via $unwind

---

Combined Use with $lookup + $filter

Suppose you do a $lookup to bring in an array of transactions for a user, but only want to keep those where amount > 100.

{
  $lookup: {
    from: "transactions",
    localField: "_id",
    foreignField: "userId",
    as: "allTransactions"
  }
},
{
  $addFields: {
    highValueTransactions: {
      $filter: {
        input: "$allTransactions",
        as: "txn",
        cond: { $gt: ["$$txn.amount", 100] }
      }
    }
  }
}

Now, each user doc contains only high-value transactions inside highValueTransactions.

---

✅ Summary of $filter

Feature	Description
Target	Works directly on arrays inside documents
Use Case	Selective element retention without $unwind
Performance	Efficient when filtering in-place
Compatible with	$lookup, $project, $addFields

---

Real-World Example Combining These Stages

Suppose you want to generate a dashboard showing:

• Total sales per item
• Top customers
• Price range distribution

db.orders.aggregate([
  { $unwind: "$items" },
  {
    $lookup: {
      from: "products",
      localField: "items.productId",
      foreignField: "_id",
      as: "productDetails"
    }
  },
  { $unwind: "$productDetails" },
  {
    $facet: {
      salesPerItem: [
        { $group: { _id: "$productDetails.name", totalSold: { $sum: "$items.quantity" } } }
      ],
      topCustomers: [
        { $group: { _id: "$customerId", totalSpent: { $sum: "$items.totalPrice" } } },
        { $sort: { totalSpent: -1 } },
        { $limit: 5 }
      ],
      priceDistribution: [
        {
          $bucket: {
            groupBy: "$productDetails.price",
            boundaries: [0, 50, 100, 150],
            default: "150+",
            output: { count: { $sum: 1 } }
          }
        }
      ]
    }
  }
])

This single aggregation query returns three different insights in one API call.

---

Conclusion

MongoDB’s advanced aggregation stages like $unwind, $lookup, $facet, and $bucket give you the ability to handle deeply structured data, join across collections, and build dashboards with a single pipeline. Mastering these techniques is essential for building powerful backend APIs and data-heavy applications.

---

Table of Contents

1. Introduction to Aggregation Stages
2. $match Stage – Filtering Documents
3. $project Stage – Reshaping Documents
4. $group Stage – Grouping and Aggregating
5. $sort Stage – Ordering the Output
6. $limit Stage – Reducing the Output Size
7. Combining Stages in a Real-World Example
8. Conclusion

---

Introduction to Aggregation Stages

MongoDB’s Aggregation Pipeline consists of multiple stages, where each stage processes input documents and passes the result to the next stage. These stages allow for powerful transformations and computations directly within the database.

Five foundational stages in most aggregation pipelines are:

• $match: Filter documents.
• $project: Include, exclude, or transform fields.
• $group: Aggregate data.
• $sort: Order results.
• $limit: Restrict the number of results.

Let’s break down each one.

---

$match Stage – Filtering Documents

The $match stage acts as a filter, similar to the WHERE clause in SQL. It passes only those documents that match the specified criteria.

Syntax:

{ $match: { field: value } }

Example:

db.orders.aggregate([
  { $match: { status: "shipped" } }
])

This filters documents where status is "shipped".

✅ Pro Tip: Place $match as early as possible in the pipeline to minimize the number of documents passed to later stages. This improves performance.

---

$project Stage – Reshaping Documents

The $project stage is used to include, exclude, or transform fields in the result set. It’s often used to:

• Rename fields.
• Create new computed fields.
• Hide sensitive or unnecessary data.

Syntax:

{ $project: { field1: 1, field2: 1, _id: 0 } }

Example:

db.orders.aggregate([
  { $project: { customerId: 1, amount: 1, _id: 0 } }
])

This outputs only customerId and amount, excluding _id.

Transform fields example:

{ $project: { fullName: { $concat: ["$firstName", " ", "$lastName"] } } }

---

$group Stage – Grouping and Aggregating

The $group stage is one of the most powerful stages in the pipeline. It’s used to group documents by a specified identifier and then apply aggregation operators such as:

• $sum
• $avg
• $min / $max
• $first / $last
• $push / $addToSet

Syntax:

{ $group: { _id: "$field", total: { $sum: "$amount" } } }

Example:

db.orders.aggregate([
  { $group: { _id: "$customerId", totalSpent: { $sum: "$amount" } } }
])

Groups orders by customerId and calculates total amount spent.

Grouping by a constant:

{ $group: { _id: null, totalRevenue: { $sum: "$amount" } } }

This aggregates across all documents.

---

$sort Stage – Ordering the Output

The $sort stage sorts documents based on specified fields.

Syntax:

{ $sort: { field: 1 } }   // Ascending
{ $sort: { field: -1 } }  // Descending

Example:

db.orders.aggregate([
  { $sort: { amount: -1 } }
])

Sorts orders by amount in descending order.

✅ Important: $sort can be resource-intensive. Ensure you use indexes when sorting on large collections.

---

$limit Stage – Reducing the Output Size

The $limit stage restricts the number of documents passed to the next stage or returned to the client.

Syntax:

{ $limit: number }

Example:

db.orders.aggregate([
  { $sort: { amount: -1 } },
  { $limit: 5 }
])

Returns the top 5 orders with the highest amount.

This stage is commonly used for pagination or leaderboards.

---

Combining Stages in a Real-World Example

Let’s imagine a sales dashboard where we need to display the top 3 customers by total purchase amount:

db.orders.aggregate([
  { $match: { status: "completed" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } },
  { $limit: 3 },
  { $project: { _id: 0, customerId: "$_id", total: 1 } }
])

Explanation:

1. Filter only completed orders.
2. Group by customer and calculate total.
3. Sort totals in descending order.
4. Limit to top 3 customers.
5. Reshape the final output.

---

Conclusion

The aggregation pipeline stages $match, $project, $group, $sort, and $limit form the backbone of most real-world MongoDB aggregation operations. When used together, they allow you to filter, transform, group, and summarize data efficiently.