Big mystery: a document was not inserted [MongoDB]

Hi all,

I have been using my app for about a year without any problems. Today, I suddenly found out that I received a transactional email that someone booked a workshop using my app (every booking = notification via email). When I tried to find that booking in my app/db I couldn’t! Now I am quite desperate and have no idea how this is possible (no error logs).

So now I’m wondering, is it possible that MongoDB sometimes fails? Can I trust MongoDB for the future? Or is it a specific Meteor problem? How should I tackle this? Every booking missed booking = losing sales.

Thanks for your advice.

Roel

It might be that mongodb (writeconcern) is to blame, but it is also possible that you have misarchitected that part of your app where you send out emails prematurely.

Shhhh. Mongodb is reliable! Or so several people have shouted at me.

Well there are some workloads which momgodb is great at and some that it is not but overall, it works fine as long as you know what your mongodb design/configuration options are.

So there’s no need to be religious about it :slight_smile:

Hehe religious…

I am doing an insert without a callback so that it would be blocking and throw an exception in case it failed. After that, I emit an event -> listener sends notifications.

Would that be a correct flow, or am I missing something significant?

Thanks guys.

Hm, where/how do you emit that event? Do you actually check that the insert returns an id?

Wierd, cause AFAIK Meteor sets the write concern to 1, thus the availability of the blocking request, which waits until the database acknowledges the write.

In the docs:

On the server, if you don’t provide a callback, then insert blocks until the database acknowledges the write, or throws an exception if something went wrong.

As per this, you should have your data.

Could we see the code? That could give more insight.

Thanks for your help guys.
To clarify things, here is my code (coffeescript + server/client):

My method to place a booking:

Meteor.methods
  placeBooking: (doc) ->
    event = Events.findOne doc.eventId
    unless event?
      throw new Meteor.Error 'no-event', 'Geen evenement gevonden.'
    # pluck the referrerId, we want to check the validity on the server only
    referrerId = if doc.referrerId? then doc.referrerId else undefined
    doc.referrerId = undefined
    # Check whether the event hasn't been fully booked yet
    if event.numberOfParticipants >= event.maxSpots
      throw new Meteor.Error 'event-fully-booked', 'Dit evenement is volzet.'
    # Bookings are not allowed for locked events
    if event.locked? and event.locked is true
      throw new Meteor.Error 'locked-event', 'Een vergrendeld evenement kan niet worden geboekt.'
    # sanitize formData
    doc = sanitizeFormData doc
    # Persist docs & send email
    bookingId = Bookings.insert doc
    _.extend doc, _id: bookingId # add the generated _id to the booking doc
    city = event.city()
    unless @isSimulation
      # check if there is a valid refferal discount
      if referrerId?
        try doc = Meteor.call 'applyDiscountCode', bookingId, referrerId
      # denormalize the total number of participants
      Events.update event._id,
        $inc: numberOfParticipants: 1
      EventEmitter.emit 'newBooking', bookingId, event, city
    return _id: bookingId, citySlug: city.slug, paymentUrl: paymentUrl

And then this is my listener code (server only):

EventEmitter.on 'newBooking', (bookingId, event, city) ->
  Meteor.defer ->
    booking = Bookings.findOne bookingId
    location = event.location()
    # Send sms to booker
    smsMessage = compileText Settings.get('bookingConfirmationSms'), booking.getTextVars()
    Sms.send booking.phone, smsMessage
    # Send all notification emails
    Emailer.send booking.email, 'Inschrijving', booking.bookingConfirmationMail()
    # Notify Slack
    if Settings.get 'useSlack'
      Slack.send
        text: "Nieuwe inschrijving!"
        fields:
          'Naam': booking.fullName
          'E-mail': booking.email
          'Gsm': booking.phone
          'Bedrag': formatMoney booking.price, 2
          'Leren': booking.heardAboutUsVia
          'Evenement': event.name
          'Stad': city.name
          'Link': "#{Meteor.absoluteUrl()}admin/inschrijvingen/#{booking._id}"
    else
      # Send email to admin
      Emailer.send Settings.get('emailAddress'), 'Nieuwe Inschrijving', booking.bookingConfirmationMail()

I think you have to save the result to a variable, to make it blocking. Also you are not checking if the update returned positive. The update returns number of affected documents (0 -> 99999), you should check for that.

bookingId = Bookings.insert doc does return so unless there’s a reason for the event emitter to run although there has been an error (thrown) this should have gone through.

What does the email you received contain? What does booking.bookingConfirmationMail() contain? Does the email look right? If it does, is there a possibility that the app itself deletes a record?

Otherwise, you might actually have been hit by a writeconcern problem.

That function compiles a text with placeholders using the booking context, eg.:
From Please pay {{price}}... ----> Please pay 219$...

The formatting of the mail is perfect.

I guess I should just hope it won’t happen too often? Would it be safer to use a service like compose or mongolab?

Alright it definitely seems like a writeconcern issue now that we know the necessary values from the order have been passed to your email context, based on the (assumed by meteor to have been) inserted order.

It does not matter where you host your mongodb, you just have to make sure that when mongodb acknowledges a database write, it has actually been persisted for good.

This is a configuration option you set globally on your database, or per insert if you pass in the necessary options.

It would be safer to set it globally, though.