---

Table of Contents

1. Introduction
2. Why Use Aggregation for Analytics?
3. Common Stages in Analytics Pipelines
4. Designing a Real-World Analytics Pipeline
5. Step-by-Step Example: E-commerce Sales Dashboard
6. Best Practices for Analytics Aggregations
7. Performance Tips
8. Conclusion

---

Introduction

In modern applications, analytics and reporting are crucial for understanding user behavior, product performance, and business trends. MongoDB’s Aggregation Framework is powerful enough to perform real-time data analytics, even over large collections, without exporting data to external systems.

---

Why Use Aggregation for Analytics?

MongoDB’s aggregation pipeline allows you to:

• Group, sort, and filter large datasets efficiently.
• Perform calculations like averages, totals, and percentages.
• Join data from other collections.
• Shape and transform your output for dashboards or APIs.
• Embed complex logic into a single query.

This enables MongoDB to act as both a transactional and analytical database for many applications.

---

Common Stages in Analytics Pipelines

Here are the most frequently used aggregation stages in analytics use cases:

Stage	Purpose
$match	Filter documents for specific time periods or users
$group	Summarize data by category, date, user, etc.
$project	Reshape documents, compute derived fields
$sort	Sort analytics results (e.g., top 10 products)
$count	Count the number of documents in a subset
$bucket	Group by value ranges (age groups, price ranges)
$facet	Run multiple aggregations in parallel
$lookup	Join data across collections
$filter	Filter array fields before further aggregation

---

Designing a Real-World Analytics Pipeline

Suppose you’re building a sales dashboard. Some key analytics requirements might be:

• Daily sales totals
• Most sold products
• Average order value
• User purchase frequency
• Time-based trends

To support this, you need an aggregation pipeline that processes data efficiently from your orders collection.

---

Step-by-Step Example: E-commerce Sales Dashboard

Collection: orders

{
  "_id": ObjectId("..."),
  "userId": ObjectId("..."),
  "items": [
    { "productId": "p1", "quantity": 2, "price": 150 },
    { "productId": "p2", "quantity": 1, "price": 200 }
  ],
  "total": 500,
  "createdAt": ISODate("2024-03-01T12:00:00Z")
}

---

Example: Get Daily Sales Summary

db.orders.aggregate([
  {
    $match: {
      createdAt: {
        $gte: ISODate("2024-03-01T00:00:00Z"),
        $lt: ISODate("2024-04-01T00:00:00Z")
      }
    }
  },
  {
    $group: {
      _id: { $dateToString: { format: "%Y-%m-%d", date: "$createdAt" } },
      totalRevenue: { $sum: "$total" },
      orderCount: { $sum: 1 }
    }
  },
  {
    $project: {
      date: "$_id",
      totalRevenue: 1,
      orderCount: 1,
      avgOrderValue: { $divide: ["$totalRevenue", "$orderCount"] }
    }
  },
  { $sort: { date: 1 } }
])

---

Result:

[
  {
    "date": "2024-03-01",
    "totalRevenue": 12000,
    "orderCount": 40,
    "avgOrderValue": 300
  },
  ...
]

This pipeline:

• Filters data for March 2024
• Groups orders by date
• Calculates total revenue, order count, and average value
• Sorts the results chronologically

---

Additional Example: Top 5 Most Sold Products

db.orders.aggregate([
  { $unwind: "$items" },
  {
    $group: {
      _id: "$items.productId",
      totalSold: { $sum: "$items.quantity" },
      revenue: { $sum: { $multiply: ["$items.quantity", "$items.price"] } }
    }
  },
  { $sort: { totalSold: -1 } },
  { $limit: 5 }
])

---

Best Practices for Analytics Aggregations

• Use $project early to reduce document size.
• Use $match to filter data early and reduce processing load.
• Use indexes to optimize $match and $sort.
• Structure documents to reduce the need for $lookup if possible.
• Cache results for heavy aggregation queries when appropriate.

---

Performance Tips

Strategy	Benefit
Use compound indexes	Boost $match + $sort performance
Avoid unnecessary $lookup	Reduce latency
Use $merge or $out	Store and reuse analytics results
Batch time-consuming pipelines	Schedule as background tasks
Use Atlas Triggers or Change Streams	Generate real-time analytics

---

Conclusion

MongoDB’s Aggregation Framework allows you to build powerful, expressive analytics pipelines directly inside the database. With the right design and performance optimizations, you can deliver fast, real-time insights without additional ETL layers.

---

Updated Table of Contents

1. Introduction to Advanced Aggregation
2. $unwind – Deconstructing Arrays
3. $lookup – Performing Joins in MongoDB
4. $facet – Multi-Faceted Aggregation
5. $bucket – Grouping Data into Ranges
6. $filter – Filtering Arrays in Aggregation
7. Real-World Example Combining These Stages
8. Conclusion

---

Introduction to Advanced Aggregation

MongoDB’s aggregation pipeline becomes incredibly powerful when you go beyond the basics. This module covers four advanced stages that are crucial for performing complex data operations:

• $unwind: Flattens arrays.
• $lookup: Performs left outer joins.
• $facet: Allows parallel pipelines for diverse analysis.
• $bucket: Groups data by value ranges.

Let’s explore each with examples.

---

$unwind – Deconstructing Arrays

The $unwind stage breaks an array field into multiple documents, one for each element.

Syntax:

{ $unwind: "$arrayField" }

Example:

db.orders.aggregate([
  { $unwind: "$items" }
])

If an order document has an items array:

{
  orderId: 1,
  items: ["pen", "notebook", "eraser"]
}

After $unwind, it becomes:

{ orderId: 1, items: "pen" }
{ orderId: 1, items: "notebook" }
{ orderId: 1, items: "eraser" }

✅ Use Case: Required when calculating statistics per array item (e.g., each item sold).

---

$lookup – Performing Joins in MongoDB

The $lookup stage is MongoDB’s way to perform SQL-style joins.

Syntax:

{
  $lookup: {
    from: "collectionToJoin",
    localField: "fieldInCurrentCollection",
    foreignField: "fieldInOtherCollection",
    as: "joinedData"
  }
}

Example:

db.orders.aggregate([
  {
    $lookup: {
      from: "customers",
      localField: "customerId",
      foreignField: "_id",
      as: "customerDetails"
    }
  }
])

Each order will now include customer info in an array field customerDetails.

✅ Pro Tip: Use $unwind on the joined field to flatten it into a single object if each order is tied to one customer.

---

$facet – Multi-Faceted Aggregation

The $facet stage allows you to run multiple aggregation pipelines in parallel and return results in one document. This is useful for analytics dashboards.

Syntax:

{
  $facet: {
    pipeline1Name: [ /* stages */ ],
    pipeline2Name: [ /* stages */ ],
    ...
  }
}

Example:

db.products.aggregate([
  {
    $facet: {
      priceStats: [
        { $group: { _id: null, avgPrice: { $avg: "$price" }, maxPrice: { $max: "$price" } } }
      ],
      categoryCount: [
        { $group: { _id: "$category", count: { $sum: 1 } } }
      ]
    }
  }
])

Returns a single document like:

{
  "priceStats": [{ "avgPrice": 35.5, "maxPrice": 100 }],
  "categoryCount": [{ "_id": "Books", "count": 10 }, ...]
}

✅ Use Case: Ideal for analytics, reporting, or dashboard queries.

---

$bucket – Grouping Data into Ranges

The $bucket stage groups documents based on specified boundaries of a field (like age or price).

Syntax:

{
  $bucket: {
    groupBy: "$price",
    boundaries: [0, 50, 100, 150],
    default: "Others",
    output: {
      count: { $sum: 1 },
      products: { $push: "$name" }
    }
  }
}

Example:

db.products.aggregate([
  {
    $bucket: {
      groupBy: "$price",
      boundaries: [0, 50, 100],
      default: "Expensive",
      output: {
        count: { $sum: 1 },
        items: { $push: "$name" }
      }
    }
  }
])

This groups products into price ranges: 0–49, 50–99, and the rest as "Expensive".

✅ Use Case: Great for histogram-style data like age brackets, price ranges, etc.

$filter – Filtering Arrays in Aggregation

The $filter operator allows you to return only specific elements from an array that match a certain condition. It’s extremely useful when you don’t want to unwind the array, but only want relevant values retained.

Syntax:

{
  $filter: {
    input: "<arrayField>",
    as: "<variableName>",
    cond: { <condition expression using the variable> }
  }
}

Example: Filter completed tasks only

Assume we have a tasks array inside user documents:

{
  name: "Alice",
  tasks: [
    { title: "Task 1", completed: true },
    { title: "Task 2", completed: false },
    { title: "Task 3", completed: true }
  ]
}

We can filter only the completed tasks using:

db.users.aggregate([
  {
    $project: {
      name: 1,
      completedTasks: {
        $filter: {
          input: "$tasks",
          as: "task",
          cond: { $eq: ["$$task.completed", true] }
        }
      }
    }
  }
])

Result:

{
  name: "Alice",
  completedTasks: [
    { title: "Task 1", completed: true },
    { title: "Task 3", completed: true }
  ]
}

---

✅ Use Cases of $filter:

• Show only high-rated products from an embedded reviews array
• Return users who have only completed certain badges or certifications
• Simplify array-based filtering without flattening via $unwind

---

Combined Use with $lookup + $filter

Suppose you do a $lookup to bring in an array of transactions for a user, but only want to keep those where amount > 100.

{
  $lookup: {
    from: "transactions",
    localField: "_id",
    foreignField: "userId",
    as: "allTransactions"
  }
},
{
  $addFields: {
    highValueTransactions: {
      $filter: {
        input: "$allTransactions",
        as: "txn",
        cond: { $gt: ["$$txn.amount", 100] }
      }
    }
  }
}

Now, each user doc contains only high-value transactions inside highValueTransactions.

---

✅ Summary of $filter

Feature	Description
Target	Works directly on arrays inside documents
Use Case	Selective element retention without $unwind
Performance	Efficient when filtering in-place
Compatible with	$lookup, $project, $addFields

---

Real-World Example Combining These Stages

Suppose you want to generate a dashboard showing:

• Total sales per item
• Top customers
• Price range distribution

db.orders.aggregate([
  { $unwind: "$items" },
  {
    $lookup: {
      from: "products",
      localField: "items.productId",
      foreignField: "_id",
      as: "productDetails"
    }
  },
  { $unwind: "$productDetails" },
  {
    $facet: {
      salesPerItem: [
        { $group: { _id: "$productDetails.name", totalSold: { $sum: "$items.quantity" } } }
      ],
      topCustomers: [
        { $group: { _id: "$customerId", totalSpent: { $sum: "$items.totalPrice" } } },
        { $sort: { totalSpent: -1 } },
        { $limit: 5 }
      ],
      priceDistribution: [
        {
          $bucket: {
            groupBy: "$productDetails.price",
            boundaries: [0, 50, 100, 150],
            default: "150+",
            output: { count: { $sum: 1 } }
          }
        }
      ]
    }
  }
])

This single aggregation query returns three different insights in one API call.

---

Conclusion

MongoDB’s advanced aggregation stages like $unwind, $lookup, $facet, and $bucket give you the ability to handle deeply structured data, join across collections, and build dashboards with a single pipeline. Mastering these techniques is essential for building powerful backend APIs and data-heavy applications.

---

Table of Contents

1. Introduction to Aggregation Stages
2. $match Stage – Filtering Documents
3. $project Stage – Reshaping Documents
4. $group Stage – Grouping and Aggregating
5. $sort Stage – Ordering the Output
6. $limit Stage – Reducing the Output Size
7. Combining Stages in a Real-World Example
8. Conclusion

---

Introduction to Aggregation Stages

MongoDB’s Aggregation Pipeline consists of multiple stages, where each stage processes input documents and passes the result to the next stage. These stages allow for powerful transformations and computations directly within the database.

Five foundational stages in most aggregation pipelines are:

• $match: Filter documents.
• $project: Include, exclude, or transform fields.
• $group: Aggregate data.
• $sort: Order results.
• $limit: Restrict the number of results.

Let’s break down each one.

---

$match Stage – Filtering Documents

The $match stage acts as a filter, similar to the WHERE clause in SQL. It passes only those documents that match the specified criteria.

Syntax:

{ $match: { field: value } }

Example:

db.orders.aggregate([
  { $match: { status: "shipped" } }
])

This filters documents where status is "shipped".

✅ Pro Tip: Place $match as early as possible in the pipeline to minimize the number of documents passed to later stages. This improves performance.

---

$project Stage – Reshaping Documents

The $project stage is used to include, exclude, or transform fields in the result set. It’s often used to:

• Rename fields.
• Create new computed fields.
• Hide sensitive or unnecessary data.

Syntax:

{ $project: { field1: 1, field2: 1, _id: 0 } }

Example:

db.orders.aggregate([
  { $project: { customerId: 1, amount: 1, _id: 0 } }
])

This outputs only customerId and amount, excluding _id.

Transform fields example:

{ $project: { fullName: { $concat: ["$firstName", " ", "$lastName"] } } }

---

$group Stage – Grouping and Aggregating

The $group stage is one of the most powerful stages in the pipeline. It’s used to group documents by a specified identifier and then apply aggregation operators such as:

• $sum
• $avg
• $min / $max
• $first / $last
• $push / $addToSet

Syntax:

{ $group: { _id: "$field", total: { $sum: "$amount" } } }

Example:

db.orders.aggregate([
  { $group: { _id: "$customerId", totalSpent: { $sum: "$amount" } } }
])

Groups orders by customerId and calculates total amount spent.

Grouping by a constant:

{ $group: { _id: null, totalRevenue: { $sum: "$amount" } } }

This aggregates across all documents.

---

$sort Stage – Ordering the Output

The $sort stage sorts documents based on specified fields.

Syntax:

{ $sort: { field: 1 } }   // Ascending
{ $sort: { field: -1 } }  // Descending

Example:

db.orders.aggregate([
  { $sort: { amount: -1 } }
])

Sorts orders by amount in descending order.

✅ Important: $sort can be resource-intensive. Ensure you use indexes when sorting on large collections.

---

$limit Stage – Reducing the Output Size

The $limit stage restricts the number of documents passed to the next stage or returned to the client.

Syntax:

{ $limit: number }

Example:

db.orders.aggregate([
  { $sort: { amount: -1 } },
  { $limit: 5 }
])

Returns the top 5 orders with the highest amount.

This stage is commonly used for pagination or leaderboards.

---

Combining Stages in a Real-World Example

Let’s imagine a sales dashboard where we need to display the top 3 customers by total purchase amount:

db.orders.aggregate([
  { $match: { status: "completed" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } },
  { $limit: 3 },
  { $project: { _id: 0, customerId: "$_id", total: 1 } }
])

Explanation:

1. Filter only completed orders.
2. Group by customer and calculate total.
3. Sort totals in descending order.
4. Limit to top 3 customers.
5. Reshape the final output.

---

Conclusion

The aggregation pipeline stages $match, $project, $group, $sort, and $limit form the backbone of most real-world MongoDB aggregation operations. When used together, they allow you to filter, transform, group, and summarize data efficiently.

---

Table of Contents

1. What is the MongoDB Aggregation Framework?
2. Why Use Aggregation in MongoDB?
3. Understanding the Aggregation Pipeline
4. Basic Aggregation Example
5. Key Aggregation Stages
6. Aggregation vs Map-Reduce
7. Performance Considerations
8. Conclusion

---

What is the MongoDB Aggregation Framework?

The MongoDB Aggregation Framework is a powerful set of tools that allows you to process data records and return computed results. It is particularly useful for data transformation and analytics, such as grouping, filtering, projecting, and calculating values based on data stored in collections.

Aggregation in MongoDB is conceptually similar to SQL’s GROUP BY clause, but with more flexibility and modularity.

---

Why Use Aggregation in MongoDB?

MongoDB’s aggregation framework helps developers:

• Perform real-time analytics directly on data stored in the database.
• Replace complex data processing in the application layer with database-side processing.
• Build dashboards, reports, and custom views efficiently.

Use cases include:

• Calculating total revenue grouped by product.
• Generating user activity statistics.
• Filtering and transforming nested documents for UI display.

---

Understanding the Aggregation Pipeline

The aggregation framework works using a pipeline approach. This means documents from a collection pass through multiple stages, each transforming the data in some way.

Think of it as an assembly line:
Each stage takes in documents, processes them, and passes them to the next stage.

Syntax:

db.collection.aggregate([
  { stage1 },
  { stage2 },
  ...
])

For example:

db.orders.aggregate([
  { $match: { status: "completed" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } }
])

This aggregates orders by customerId and returns the total amount spent per customer for completed orders.

---

Basic Aggregation Example

Let’s say you have a sales collection:

{
  "_id": ObjectId("..."),
  "region": "North",
  "amount": 100,
  "product": "Book"
}

You want to calculate the total sales per region:

db.sales.aggregate([
  { $group: { _id: "$region", totalSales: { $sum: "$amount" } } }
])

Output:

[
  { "_id": "North", "totalSales": 5000 },
  { "_id": "South", "totalSales": 3000 }
]

---

Key Aggregation Stages

MongoDB provides many stages for pipelines. Some of the most commonly used include:

Stage	Description
$match	Filters documents (like WHERE in SQL).
$group	Groups documents and performs aggregations ($sum, $avg, etc).
$project	Reshapes each document (like SELECT clause).
$sort	Sorts documents.
$limit	Limits the number of output documents.
$skip	Skips a specific number of documents.
$unwind	Deconstructs arrays for processing.
$lookup	Joins documents from another collection.

Each stage returns documents to be used by the next stage, making the pipeline modular and flexible.

---

Aggregation vs Map-Reduce

MongoDB also offers Map-Reduce, a powerful feature for custom aggregations. However, it’s often less performant and more complex than the aggregation framework.

Feature	Aggregation Framework	Map-Reduce
Performance	Faster, optimized	Slower
Syntax	Easier to write	More complex (requires JS functions)
Use Cases	Most aggregations	Custom logic not supported by aggregation

In most real-world applications, the aggregation pipeline is preferred over Map-Reduce.

---

Performance Considerations

When using aggregation, keep these tips in mind:

• Index usage: The $match stage benefits from indexes.
• $project early: If fields are not needed, exclude them early with $project.
• Avoid large $lookup operations unless necessary.
• Use $facet for multi-faceted aggregations in dashboards.
• Use $merge or $out to store results when needed.

MongoDB has built-in explain plans to analyze aggregation performance.

---

Conclusion

The MongoDB Aggregation Framework is a cornerstone for building powerful data-processing pipelines directly within your database layer. Whether you’re building reports, dashboards, or simply need to transform data on the fly, understanding how aggregation pipelines work is crucial.

In the next modules, we’ll dive deeper into individual stages like $match, $group, $project, and explore advanced techniques like joins with $lookup, and multi-stage processing.