I am writing a server method which needs to run a large aggregate query and process the results. The query runs fine in Robo3T but in Meteor I get the following error returned:
aggregation result exceeds maximum document size (16MB)
What are my options? Am I able to get the query to return a cursor rather than an array, or do I need to add in $skip and $limit and run the query in chunks?
(I use the meteorhacks:aggregate package to allow me to run aggregate queries).
You could output the result to a temporary collection and then read from it.
https://docs.mongodb.com/manual/reference/operator/aggregation/out/
EDIT: I also believe you could also use “collection.rawCollection()” and to get the native node.js driver and use it to consume the cursor.
Thank you.
Outputting to a temporary collection all seems a bit messy, but I will definitely take a look at using the native driver to see how easy it is to consume the cursor.
That looks like a limitation of the meteorhacks:aggregate package using an old form of the aggregation method.
As @workman says you should really be using rawCollection() to expose the underlying node driver to do aggregations. In my opinion, it’s no harder than using a package - and there’s no legacy, unmaintained package lock-in.