Aggregation document size > 16mb


#1

I am writing a server method which needs to run a large aggregate query and process the results. The query runs fine in Robo3T but in Meteor I get the following error returned:

aggregation result exceeds maximum document size (16MB)

What are my options? Am I able to get the query to return a cursor rather than an array, or do I need to add in $skip and $limit and run the query in chunks?

(I use the meteorhacks:aggregate package to allow me to run aggregate queries).


#2

You could output the result to a temporary collection and then read from it.

https://docs.mongodb.com/manual/reference/operator/aggregation/out/

EDIT: I also believe you could also use “collection.rawCollection()” and to get the native node.js driver and use it to consume the cursor.


#3

Thank you.

Outputting to a temporary collection all seems a bit messy, but I will definitely take a look at using the native driver to see how easy it is to consume the cursor.


#4

That looks like a limitation of the meteorhacks:aggregate package using an old form of the aggregation method.

As @workman says you should really be using rawCollection() to expose the underlying node driver to do aggregations. In my opinion, it’s no harder than using a package - and there’s no legacy, unmaintained package lock-in.