Scheduling Emails

help
21

I do agree with you with marketing/bulk emails where no home-grown
solution be it meteor or else would provide any benefit.

But for transactional emails,

a) offloading a message to another thing requires you to compile the
building blocks of the message and pass on to that thing using its API.

b) sending a message directly from with meteor requires you to compile
the building blocks of the message and pass on to that thing using SMTP.

And both would be non-blocking. So what would be the benefit, in your
opinion, to take emailing out and create a separate stack?

The way I see it is unnecessary complication.

But I do sincerely not want to sound like a wise-ass here and I do want
to know your reasons.

Thanks

22

In either A or B, I would only pass the fields and their values to the other process, which may easily be an external service like Mandrill or an internal cron job.

Anything that uses CPU cycles is a blocking operation, no matter how small. Anything that is put into the ‘work queue’ so to speak is a blocking operation. Because the main thread works by going through that work queue. Whether it is ten emails a day or 100 emails a day, something that goes into the main work queue is extra load. You may think this over-optimization, but having seen so many projects implementing sending of transactional emails as part of the core web app and failing, I would prefer to be cautious and spend an extra day implementing a separate worker for emails to sending them from the main app to gain a few hours of development time.

A web application should not be responsible for sending emails, unless we are talking about a very small application. This has nothing to do with Meteor’s superior work model or sending emails being a non-blocking operation. Meteor’s only responsibility should be serving clients on the web front. (Logins with an external service is a different subject altogether.)

Here is how I would implement it.

In Meteor main app, any email to be sent is written to ‘email_queue’. Only the fields, recipients _id, and which template to use. Not the HTML itself, just the _id of the template.

The other thread has access to user info and the templates. And works on that data. If it’s not using an external service like Mandrill, it generates the HTML and sends the email itself. It takes into account all possibilities like mailbox full, recipient mailbox does not exist.

I have a quick question that I don’t know the answer to. Let’s say the recipient server returned mailbox full. How do you proceed? Or the recipient server is down. What happens in case of a timeout? How do you handle those? I’m not trying to prove anything, just curious as to the actual process of sending the emails from Meteor. Because, you know, there may be problems in virtually every step.

23

Regarding the question, I do:

  1. this.unblock() so that I don’t actually wait for the email method to return.
  2. I use the email method’s callback to look for an error object and reschedule as necessary

Regarding your overall answer; first things first, thank you for taking the time to explain it so thoroughly!

I agree with you on a broader basis where if the app is one that you are looking to scale, then your overall premise should be applied to all parts of the application in terms of a (micro)service architecture where all and any part of your application is a confined piece of code that takes an input and reports back to a backbone service orchestrator.

But hey, I’m coming from a javaee world where I’ve seen such architecture overapplied and utterly abused one too many times.

The “it should be able to scale” pseudo-requirement causes hello world apps to take two weeks to print out the world without the hello.

So my take is, start out with fully-confined apps (except when you have to compartmentalize) and iterate as you go along. I think of taking out email sending code into a separate worker as a feature that should be prioritiezed among a broader list of features including that shiny call to action button on the top left corner that the marketing guy asked for.

So in essence, we actually agree :smile: