How do you have implemented event-based email campaigns in your app?

[a4xrbj1]

First a quick definition what I mean by “event-based” email campaigns:

We want to trigger a specific email when certain conditions are fulfilled (can be one or more conditions). To give some examples:

• user is stuck on the account verification step (in our app) for x number of days
• it’s been y days since event z happened and user hasn’t done action abc

Those campaigns can be single-step or multi-step. We have to take care that events don’t overlap and users get 2 different emails from their actions/non-actions!

It obviously doesn’t make any sense to code an expensive event-based campaign management solution like Unica which I have used in previous jobs before.

Has anyone of you been in the same situation where you wanted to increase the engagement of your app’s users with a well designed system of event-based campaigns?

A quick search on NPM didn’t reveal anything, this is a backend operation anyway.

The (painful) alternative would be to use jobs that run at regular time intervals and hard-code the conditions vs fields from collections. Not ideal and will surely get very complicated very quickly.

Thanks in advance for sharing comments and ideas!

[paulishca]

I am starting from this particular line: " * it’s been y days since event z happened and user hasn’t done action abc". If you used a Cron service, you would actually run 1 job a day or split of multiple jobs if you really have a lot of users to email to. You would run multiple jobs eventually if you want to target particular times of the day (e.g. business hours rather than sending an email at midnight).
For such things I’ve been using https://github.com/percolatestudio/meteor-synced-cron and the API looks like this (server side indeed):

Meteor.startup(function () {
  // code to run on server at startup
  SyncedCron.start()
  // emailAnnouncement("submission ended", contest._Id)
  /* Stop jobs after 15 seconds
   Meteor.setTimeout(function () {
   SyncedCron.stop()
   }, 15 * 1000)
   */
})

// optionally set the collection's name that synced cron will use
SyncedCron.config({
  // Log job run details to console
  log: true,

  // Name of collection to use for synchronisation and logging
  collectionName: 'cronHistory',

  // Default to using localTime
  utc: false,

  /*
   TTL in seconds for history records in collection to expire
   NOTE: Unset to remove expiry but ensure you remove the index from
   mongo by hand

   ALSO: SyncedCron can't use the `_ensureIndex` command to modify
   the TTL index. The best way to modify the default value of
   `collectionTTL` is to remove the index by hand (in the mongo shell
   run `db.cronHistory.dropIndex({startedAt: 1})`) and re-run your
   project. SyncedCron will recreate the index with the updated TTL.
   */
  collectionTTL: 172800
})


SyncedCron.add({
  name: 'Contest second_trigger',
  schedule: function (parser) {
    // parser is a later.parse object
    return parser.text('every 2 mins')
  },
  job: function (intendedAt) {
    let contests = Contests.find({
      step: 4,
      secondTriggerActionsComplete: { $not: { $exists: true } },
      secondTriggerDate: {
        $lt: new Date()
      }
    }).fetch()
    console.log(contests.length)

    contests.map(contest => {
      Meteor.call('/app/cron/actions/second_trigger', contest._id, (err) => {
        if (err) {
          console.log(err)
        } else {
          Contests.update(contest._id, {
            $set: { secondTriggerActionsComplete: true }
          })
        }
      })
    })
    console.log(intendedAt)
  }
})

[a4xrbj1]

Thanks, I should have mentioned that we’re using simonsimcity:job-collection as the original job-collection is no longer looked after unfortunately.

So that part isn’t a problem at all, we do use it already do run regular jobs including sending out daily emails to users at their best time zone match.

The question was more about the setup of event-based (and by that I don’t mean only time-based) emails.

[paulishca]

My understanding is that both your examples are time based. What would be your event-based like? Action on click of a button or visiting a page?

[a4xrbj1]

Paul, the second example has a certain event in it that will be checked as well. There is indeed a lot of concentration on time passed since something as I want to drive engagement up with these sorts of event based emails.

But to give you other examples:

• user hasn’t logged in for x days
• user has y new private messages or responses to discussions waiting unread
• user hasn’t uploaded or worked on his own family tree
• user hasn’t uploaded or worked on one of his DNA matches family tree

So the events are all based on backend (meaning data stored in MongoDb) and not on frontend interactions (as even they trigger insert/update in docs we keep about user behavior).

Hope this helps,

Andreas

[raphaelarias]

We trigger Hubspot workflows or Hubspot contact properties updates. This allow us to define the actions on Hubspot instead of in the codebase.

For cron we use AWS Lambda (or now AppEngine cron).

For events, we use Google Cloud’s PubSub, which queue the events, which are then processed in a seperate back-end-only Nodejs app.

[a4xrbj1]

Thanks, this is an interesting approach. Will check out the various products to see if it is a feasible solution for us too.

[paulishca]

When you mention “events”, I think of the technical event, the Node event (https://www.freecodecamp.org/news/using-events-in-node-js-the-right-way-fc50c060f23b/)
However I personally don’t see any case where an emitter is involved unless you want, for instance to send an email on the 5th notification received by a user where I would personally use a method anyway.

What I would do, I would target the perfect time for the region (e.g. 10 AM) and this is when I would run a series of cron jobs. I am sorry but I possibly miss the point on the event concept but I see them all as time based actions.

At 10 am I run this job:

• Query the DB for “user hasn’t logged in for x days”. Unblock the action and as found call a method to email the user. Possibly do a throttling on an array of user Ids if they’re many, just to avoid a high load.
• “user has y new private messages or responses to discussions waiting unread”. There is no point to send an email at 2AM. So at 10AM I would just run a query (if more than 3 or 10 private messages pick the userId). If I’d want to send an email on the 5th unread notification, I’d make that part of the notification writing to Mongo.
  Logic: On the user document in the User Collection I save the count and in the Notifications Collection I save the notification itself. When I save a new notification I compare the count on the user document and if a condition is met, I send the email.
• “user hasn’t uploaded or worked on his own family tree” a 10AM cron job on a “createdAt” field on some collection.
  When I think programatic events, I think in terms of proactive and reactive. Cron is proactive for me, emitters are reactive. The thing is that I don’t see the emitters for your cases.

Cron jobs are best ran from a separate Meteor Box, with a simple UX for just managing and monitoring the jobs. I’d run them on a secondary MongoDB server if (you use replicas).

I really don’t want to annoy you but would love to understand and learn what you consider an “event based” and why you feel cron is not appropriate for this.

[hemalr87]

Your use case seems a good match for a service like Intercom or customer.io (or any of the alternatives) - there are quite a few of them out there now. I do understand that that can get quite expensive - in fact, we are just in the process of moving away from Intercom because of the price.

In our case though, the events are largely in sync with Stripe events as well, so I am simply going to link up Stripe’s webhook calls to send the relevant emails out (at the expense of sending out some in app messages…).

Outside of a third party solution, the way I see it, there are two options:

1. Have a regular cron job or similar (which you and others in the thread have already mentioned) that checks at specified times/days (I think this approach would probably be the simpler option). This is what I opted for for another project where the events and conditions were too complex to use a third party like Intercom.
2. Whenever event x occurs, that should trigger event y if z, a part of the event x should be to set up its own delayed function (whether that is using synced cron or whatever…).

It’s an interesting scenario and I’ve often thought of creating a service to tackle problems like this. From what is out there, I thing iron.io service workers come closest. If not, then I imagine one of the serverless options could be configured to do something similar.

[wildhart]

I do this quite easily using a job scheduler built into my meteor code. I used to use msavin:usercache but that has a ‘wontfix’ performance bug if you define lots of different types of jobs, so now I use my own fork wildhart:meteor-jobs which is more efficient.

I have lots of different types of ‘event’ driven delayed actions. For most of these I delete any previous scheduled jobs of the same type for that user, and then schedule a new job to run after the delay.

E.g. user hasn’t logged in for x days:

// code which runs whenever a user logs in...
Jobs.clear('*', 'sendNotLoggedInRecenlyEmail', userId);
Jobs.run('sendNotLoggedInRecenlyEmail', userId, {in: {days: notLoggedInReminderDays}});

Another way is to set a flag on the user’s record recording the time when they last logged in. Then schedule a repeating job for every 4 hours which queries the Meteor.users db for users who’s lastLoggedInDate is over x days ago, and sends them all an email. I find that 4 hours is a good compromise between sending the email at a time when the user is likely to be awake (since it will be within 4 hours of a time of day when they last logged in), and minimising load on the server.

Having said that, as long as you put indexes on any fields you’ll be querying then I don’t notice any performance hit. I run all these jobs on the same host as my app and DB and I don’t see any 4 hourly spikes at all.

I use this second method every 4 hours to send emails to users who’s subscription will expire in x days.