Hi everybody.
I’m here to share my solution of utilizing Mongo DB transactions with Meteor 1.8.
And just as important - how it can be used in Meteor Methods (e.g. returning a value from a transaction to the client). I’ve learned it’s not as simple is it may sound so I wanted to share the solution, I hope it’ll be useful for others.
Side Note:
I use the term my solution not as a pride of ownership but to clarify that it’s not a well-tested solution. It’s my personal way of solving this riddle with Meteor. Hopefully with time we will have a cleaner, well-tested, standard approach to enjoy transactions in Meteor. That’s why I preferred to not wrap this into a small library and rather make sure people are aware of what’s happening under the hood so when the time comes and the technology matures in this area they can easily adapt their code.
Intro: What’s the big deal?
2 months ago I started developing a meteor project. Pretty quickly I realized that I have to use transactions in my app and how lucky I felt when I found out that Meteor’s latest version (at the time was 1.8) supports Mongo 4 and that Mongo 4 supports transactions 
! woohoo 
! Right? Well, kind of… yes. But there are a few obstacles.
The Obstacles
To begin with, Meteor collection objects (Mongo.Collection) don’t support the transactions API, which means we have to access the raw collection underneath and working directly with raw collections introduces few obstacles:
1. Missing Documentation - first practical problem is that documentation is missing around that subject and it’s challenging (at the time of writing) to find a complete, working example of how to correctly execute transaction when working with the raw collection in Meteor.
2. Different ID Generation Mechanisms - Meteor uses a different ID generation (string) than Mongo’s default (ObjectID), so using Meteor collections as well as working with its raw collections directly may lead to inconsistency in the database (objects will have different _id values if inserted both via collections and raw collections).
3. Attached SimpleSchema Doesn’t Run - this one is more specific for meteor-collection2 users. If you attach a SimpleSchema to your Meteor collection, you will not be pleased to know that it’s not attached to the raw collection. It basically means that when you’re working with raw collections - you get no object validation against the schema, no object cleaning, etc. (what you do get if you access Meteor collection).
4. Transactions in Methods Doesn’t Return Value/Error To The Client - Mongo transactions are async and you can’t run async operations in a sync Method and expect it to return its value to the client. The method will return before waiting for your async call to return.
Sorry to be the party pooper but I suggest you put the beers back in the fridge 
Am I the only one who faced all of these problems? not much information online at the time of writing.
So without further ado…
The Solutions
1. Basic Transaction Example in Meteor
import { MongoInternals } from 'meteor/mongo';
...
// finding item in collection1, inserting result to collection2
// both collection1 and collection2 are Meteor Collection objects
const transactionExample = async (collection1, selector, collection2) => {
const { client } = MongoInternals.defaultRemoteCollectionDriver().mongo;
const session = await client.startSession();
await session.startTransaction();
try {
const item = await collection1.rawCollection().findOne(selector, { session: session });
// here you may want to check item value before proceeding
const response = await collection2.rawCollection().insertOne(item, { session: session });
// here you may want to check insertion response
await session.commitTransaction();
} catch (err) {
await session.abortTransaction();
// possibly add: console.error(err.message);
} finally {
session.endSession();
}
}
2. Achieving Document ID Consistency
To ensure you have a consistent _id for all documents you have two options:
-
Option A: Mongo-like ID consistency - set Meteor to generate Mongo Object ID as
_idfor all collections, in case you’d want to use transactions and for a general consistency in your database (look for idGeneration in the docs). -
Option B: Meteor-like ID consistency - pass a string value as the
_idproperty in all raw collection insertions.
Having started with option A I advise you to go with option B 
.
The problem with option A is that when Mongo ID generation is used by Meteor it’s not really generating an identical ID object as Mongo does (i.e. the raw collection) and I found out that a Meteor-generated Object ID can’t be used as a selector when working with raw collections.
As long as a document was created with Meteor collection (i.e. Meteor generated its ID), calling:
await collection.rawCollection().findOne(item._id, { session: session });
will return nothing, no matter how you construct the selector ( { _id: item._id }, etc. ).
In fact an item containing a Meteor-generated Mongo.ObjectID _id value that is passed as a selector to any raw collection queries will actually filter the results out because of it. This probably can be reported as a bug in Meteor repo.
A cleaner solution (Option B) then would be to add a string id before every raw insertion:
const itemWithID = { ...item, _id: new Mongo.ObjectID()._str };
await collection.rawCollection().insertOne(itemWithID , { session: session });
3. Utilizing SimpleSchema - you need to clean and/or validate your objects manually before raw collection insertions:
import SimpleSchema from 'simpl-schema';
...
// optionally define your SimpleSchema with cleaning options
let cleanOptions = { filter: true, autoConvert: true, removeEmptyStrings: true, trimStrings: true };
schema = new SimpleSchema( { /* your schema defs */ } , { clean: cleanOptions });
// attach Simple Schema to a Meteor collection
collection.attachSchema(schema);
// calling collection.insert(..) will now automatically clean and validate
// when working with the raw collection, we need to clean and validate manually
...
// within a transaction, before every insertion:
// 1. clean the item (if desired)
let cleanOptions = { filter: true, autoConvert: true, removeEmptyStrings: true, trimStrings: true };
let cleanItem = await collection.simpleSchema().clean(item, { ...cleanOptions, modifier: false });
// 2. validate the clean item against the schema (fail insertion on validation failure)
await collection.simpleSchema().validate(cleanItem, { modifier: false });
// 3. adding string ID to avoid the Mongo's ObjectID generation
cleanItem = { ...cleanItem, _id: new Mongo.ObjectID()._str };
// at last, insert the clean, validated item
await collection.rawCollection().insertOne(cleanItem, { session: session });
...
4. Methods 
By now we have the transaction code sample, we have a consistent _id generation across raw collections and we can even utilize SimpleSchema.
The last missing piece in the puzzle is to be able to utilize all of this in a Method in a synchronized manner, i.e. to return a transaction value (or a transaction failure error) to the client.
Option A: Async Method + Async/Await
UPDATE: thanks to @robfallows’s input I added this option which is simpler than Option B. Unfortunately it’s not officially documented at the time of writing and I didn’t find many good examples of it except for his great article. I have used both option A and option B in production for several months now and they both work flawlessly
As @robfallows describes, you can decalre an async method and execute your transaction as you’d expect with async/await:
Meteor.methods({
async doSomething() {
// just put your transaction here
}
});
To keep our methods clean we can define a utility function that will run our mongo operations within a transaction context:
import { MongoInternals } from 'meteor/mongo';
...
// utility async function to wrap async raw mongo operations with a transaction
const runTransactionAsync = async asyncRawMongoOperations => {
// setup a transaction
const { client } = MongoInternals.defaultRemoteCollectionDriver().mongo;
const session = await client.startSession();
await session.startTransaction();
try {
// running the async operations
let result = await asyncRawMongoOperations(session);
await session.commitTransaction();
// transaction committed - return value to the client
return result;
} catch (err) {
await session.abortTransaction();
console.error(err.message);
// transaction aborted - report error to the client
throw new Meteor.Error('Database Transaction Failed', err.message);
} finally {
session.endSession();
}
};
And then use it as follows from our async method:
import { runTransactionAsync } from '/imports/utils'; // or where you defined it
Meteor.methods({
async doSomething(arg) {
// remember to check method input first
// define the operations we want to run in transaction
const asyncRawMongoOperations = async session => {
// it's critical to receive the session parameter here
// and pass it to every raw operation as shown below
const item = await collection1.rawCollection().findOne(arg, { session: session });
const response = await collection2.rawCollection().insertOne(item, { session: session });
// if Mongo or you throw an error here runTransactionAsync(..) will catch it
// and wrap it with a Meteor.Error(..) so it will arrive to the client safely
return 'whatever you want'; // will be the result in the client
};
let result = await runTransactionAsync(asyncRawMongoOperations);
return result;
}
});
Option B: Meteor.wrapAsync to the Rescue!
We are going to utilize Meteor.wrapAsync(func, [context]) in order to wrap our async transaction execution when used from within a Meteor method. Also around this I feel there is a lack of examples in the documentation however this thread is already long enough so I will not explain the not-very-intuitive syntax.
See below how I use a utility function called runTransaction to execute my raw collection opertaions. This function receives an async function with the desired operations to run, it first wraps it with another async function that sets up a transaction and runs the input operations from within that transaction. Using the wrapAsync it then executes this in a synchronized manner. Yes, I know it’s confusing.
import { MongoInternals } from 'meteor/mongo';
...
// utility function to make an async transaction synchronized
const runTransaction = asyncRawMongoOperations => {
// getting Mongo's client to setup a transaction
const { client } = MongoInternals.defaultRemoteCollectionDriver().mongo;
// define an async function named 'asyncExecution' that:
// 1. sets-up a transaction
// 2. runs asyncRawMongoOperaions function parameter within the transaction
// 3. returns asyncRawMongoOperaions results or reports a transaction error
let asyncExecution = async function(client, callback) {
let error = null, result = null;
const session = await client.startSession();
await session.startTransaction();
try {
// running the async operations
result = await asyncRawMongoOperations(session);
await session.commitTransaction();
} catch (err) {
// abort transaction and set a Meteor.Error, this will be returned to the client
await session.abortTransaction();
console.error(err.message);
error = new Meteor.Error('Database Transaction Failed', err.message);
} finally {
session.endSession();
}
callback(error, result); // return values to the client, see Meteor.wrapAsync docs
};
// make asyncExecution synchronized
let syncExecution = Meteor.wrapAsync(asyncExecution);
// run syncExecution and return result to to the caller
return syncExecution(client);
};
Then we can use this utility function to define and execute our raw mongo operations from within a Method (usage very similar to Option A but in a method which is not declared async):
import { runTransaction } from '/imports/utils'; // or where you defined it
Meteor.methods({
doSomething(arg) {
// remember to check method input first
// define our transaction operations
const asyncRawMongoOperations = async session => {
// it's critical to receive the session parameter here
// and pass it to every raw operation as shown below
const item = await collection1.rawCollection().findOne(arg, { session: session });
const response = await collection2.rawCollection().insertOne(item, { session: session });
// if Mongo or you throw an error here runTransaction(..) will catch it
// and wrap it with a Meteor.Error(..) so it will arrive to the client safely
return 'whatever you want'; // will be the result in the client
};
// where the async magic happens, synchronously ;)
return runTransaction(asyncRawMongoOperations);
}
});
Throwing Custom Error
If you use these utility function/s I posted (runTransactionAsync / runTransaction), note they catch every error thrown within the transaction but expect an object similar to Mongo Error so if you want to throw a custom error message from within your transaction code, make sure you adapt Mongo Error structure (e.g. { message: 'your error message' }) or modify the catch clause to fit your needs. Keep in mind you probably do want to catch Mongo Error there in case anything fails in Mongo.
Conclusion
Now you can go get the beers… go ahead and grab one for me too if you don’t mind 
.
If someone falls down the same rabbit hole as I did, I can share the transaction API (utility functions) that I use in order to automatically:
- clean and validate selectors and modifiers (SimpleSchema) on any insert/update.
- check Mongo’s responses after every raw operation, and throw error if needed.
- assign string
_idon every insert.
So that every time I need to use a transaction I focus only on the actual operations I need to do, avoiding all the pitfalls on the way.
P.S.
-
I didn’t run the exact code that’s pasted here because I use it from within utility functions and I preferred to share a more clear and simple version of it. If something doesn’t work feel free to let me know.
-
I will be happy if other people can help simplifying or improving these solutions or have any comments whatsoever.
Cheers.
