Blocking collection updates in a loop

Hi guys

I need to loop over an object, and upsert for each key value pair in the object.

Currently I’m doing something like

for (key in obj) {
  Collection.upsert(query, update);
//Logs after loop has finished

However, I have read in the docs Collection.upsert blocks on the server.
I am calling upsert ~7000 times. Does upsert block in this situation? Should i be doing something more like

const upsertPromises = [];
for (key in obj) {
  Collection.upsert(query, update, (err, changed) => {
    if (err) return reject(err);
.then(() => console.log('Done'))
.catch((err) => console.log(err));

Essentially, I’m checking whether the db updates are called in parallel, or sequence when in a loop.

Thanks :smiley:

Essentially, you’re correct in that MongoDB operations are done sequentially. However, you may find this post interesting if you’re interested in performance:

As far as using Promises is concerned, it would be better to make use of the underlying rawCollection() methods, as they’re Promisified already:

const upsertPromises = Objecct.keys(obj).map(key => {
  return Collection.rawCollection().update(query, update, { upsert: true });
Promise.all(upsertPromises) ...

Thanks for that link. Really insightful article, shame i couldn’t find it through my googling.

The article mentioned collection.rawCollection should be used for batch operations of any type.

Does your snippet demonstrate the correct usage?

Should be correct, but the update syntax for an upsert is from memory.

Use Mongo Bulk API for best efficiency.

Here’s the gist (untested)

const bulk = Collection.rawCollection().initializeUnorderedBulkOp()
for (...) {
Meteor.wrapAsync(bulk.execute, bulk)()

1 Like

That’s really helpful. Thanks :smiley:

Following up on this.

I tried to return Collection.rawCollection().update(query, update, { upsert: true }); in a ValidatedMethod method and got an Error: max call stack size exceeded. Collection.rawCollection().update(query, update, { upsert: true }); itself seems fine as i am able to add a then and catch chain without trouble inside the method. The error only occurs when i return the Collection.rawCollection().update(query, update, { upsert: true }); promise.

Additionally, the promise shown below returned (and worked) fine (based on my original pattern) with exactly the same query and options as the erroneous one:

Collection.upsert(query, update, options, (err, changed) => {
  if (err) return reject(err);

Any idea why this may be?