Understanding the Mongoose Aggregation Pipeline

Aggregation pipelines in MongoDB, particularly when using Mongoose, provide a powerful and flexible way to process and transform data. Each stage in the pipeline operates on the data, passing the transformed data to the next stage. This allows for complex data transformations to be performed efficiently.

Basic Concept

Aggregation pipelines are composed of stages, each of which transforms the data in some way. The output of one stage becomes the input for the next. Here's a simple example:

javascriptCopy codedb.orders.aggregate([
    // Stage 1: Filter pizza order documents by pizza size
    {
        $match: { size: "medium" }
    },
    // Stage 2: Group remaining documents by pizza name and calculate total quantity
    {
        $group: { 
            _id: "$name", 
            totalQuantity: {
                $sum: "$quantity"
            } 
        }
    }
]);

In this example, the first stage filters the orders to only those with a size of "medium". The second stage groups the remaining documents by the pizza name and calculates the total quantity for each type of pizza.

Key Aggregation Stages

1. $group

   • This stage groups documents by a specified id expression. Note that this id is not the same as the _id ObjectId assigned to each document.

    javascriptCopy codedb.listingsAndReviews.aggregate([
        { 
            $group: {
                _id: "$property_type"
            } 
        }
    ]);

2. $limit

   • Limits the number of documents passed to the next stage.

    javascriptCopy codedb.movies.aggregate([
        { $limit: 1 }
    ]);

3. $project

   • Specifies which fields to include or exclude in the output documents. This is similar to the projection used in the find() method.

    javascriptCopy codedb.restaurants.aggregate([
        {
            $project: {
                "name": 1,
                "cuisine": 1,
                "address": 1
            }
        },
        {
            $limit: 5
        }
    ]);

4. $sort

   • Sorts documents in the specified order. Use 1 for ascending order and -1 for descending order.

    javascriptCopy codedb.restaurants.aggregate([
        {
            $sort: { "accommodates": -1 }
        },
        {
            $project: {
                "name": 1,
                "cuisine": 1,
                "address": 1
            }
        },
        {
            $limit: 5
        }
    ]);

5. $match

   • Filters documents to pass only those that match the specified criteria. Using $match early in the pipeline can improve performance by reducing the number of documents that subsequent stages must process.

    javascriptCopy codedb.listingsAndReviews.aggregate([
        {
            $match: {
                "property_type": "house"
            }
        },
        { $limit: 2 },
        {
            $project: {
                "name": 1,
                "bedrooms": 1,
                "price": 1
            }
        }
    ]);

6. $addFields

   • Adds new fields to the documents.

    javascriptCopy codedb.restaurants.aggregate([
        {
            $addFields: {
                avgGrade: {
                    $avg: "$grades.score"
                }
            }
        },
        {
            $project: {
                "name": 1,
                "avgGrade": 1
            }
        },
        {
            $limit: 5
        }
    ]);

7. $count

   • Counts the total number of documents that pass through the pipeline.

    javascriptCopy codedb.restaurants.aggregate([
        {
            $match: {
                "cuisine": "indian"
            }
        },
        {
            $count: "totalIndian"
        }
    ]);

8. $lookup

   • Performs a left outer join with another collection in the same database.

    javascriptCopy codedb.comments.aggregate([
        {
            $lookup: {
                from: "movies",
                localField: "movie_id",
                foreignField: "_id",
                as: "movie_details"
            }
        },
        {
            $limit: 1
        }
    ]);

9. $out

   • Writes the output documents to a specified collection. The $out stage must be the last stage in the pipeline.

    javascriptCopy codedb.listingsAndReviews.aggregate([
        {
            $group: {
                _id: "$property_type",
                properties: {
                    $push: {
                        name: "$name",
                        accommodates: "$accommodates",
                        price: "$price"
                    }
                }
            }
        },
        {
            $out: "properties_by_type"
        }
    ]);

Conclusion

The Mongoose aggregation pipeline is a versatile tool that allows for complex data manipulation directly within the database. By understanding and utilizing various stages such as $match, $group, $project, and others, you can effectively query and transform your data to meet your application's needs. The key to mastering aggregation pipelines lies in understanding how each stage works and how to chain them together to achieve the desired result.