Aggregate $look up vs collection.find()

help
1

I have large data (maybe more than 5000000)
I have 2 options :
1 . Aggregate $look up data

 let data = ParentCol.aggregate(
   [
     {
        // Child 1
        $lookup:
        {
          from: 'child1',
          localField: '_id',
          foreignField: 'child1Id',
          as: child1Doc
         }
     },
     {
        // Child 2
        $lookup:
        {
          from: 'child2',
          localField: '_id',
          foreignField: 'child2Id',
          as: child2Doc
         }
     }
    // 2 $lookup more child have child more
     .........
   ]
)

2 . Collection.find()

let parentDoc = ParentCol.find(selector).fetch()
let child1Id = [] ,child2Id = []
// Loop push childId
for(let it of parentDoc){
    if(it.child1Id) child2Id.push(it.child1Id)
    else if(it.child2Id) child2Id.push(it.child2Id)
}
// Find Child
let child1Doc = Child1Col.find({_id:{$in:child1Id}}).fetch()
let child2Doc = Child2Col.find({_id:{$in:child2Id}}).fetch()
// Child have child and loop push Id to find child more
.........

// At the end loop combine array to parent

1 vs 2, which one is better for performance ?

2

You should really always test yourself for your own use case to find the best performance.

I think it depends more on how many documents your query will match, and you will push to the client. If it’s very little (1-20), I don’t think it matters which way you fetch them. However, when working with a large amount of matching documents, the Mongo aggregation framework can eclipse the performance of a Node server (or JS client) in terms of processing data. So you could probably achieve way better performance using aggregation as opposed to wiring the documents together like in your option 2. Also note that publishing documents also takes some time, so you want to publish as little data (fields/documents) as possible when working with larger amounts (50+) of documents on a single page.

But again, test for yourself to find out what works best.

3

When I try to test with large data on server, Option 2 is faster than option 1 , But option 2 is many time connect to server . For option 1 if aggregate for $sum with out $lookup is faster than with $lookup and option 2 if I find then loop for sum data it’s slower than option 1 . I don’t know why ?

4

Have you got indexes on all fields you $lookup?

5

Not yet, I will try it .