How to prevent insert duplicate records?

I used meteor with vue by using mongoose library with mongodb when I sometime insert on form sale it duplicate records (it mean insert once receive two recored) and sometime it insert in normal. I don’t know why? any have suggestion for prevent duplicate any more. please

I know some people they double click on everything, I may trigger multiple method calls inserting data.
You may want to prevent that action from the client.
You can also work with the server side, there are some options here, e.g: creating unique indexes, check if the same user just created a record milliseconds ago …

There are multiple ways to solve this:

Client-side
C1: Debounce your submit function: JavaScript Debounce Function
C2: Disable the submit trigger element, e.g., disable the submit button on click
C3: Add a processing layer on the form, e.g., processing animation when submitting something

Server-side
S1: Use a rate limiter for your inputs by hashing your input parameters and using them as a key to your rate limiter (dozens of npm packages)
S2: Use existing Meteor rate limiter for methods (although it is meant for DDOS-like connections): Methods | Meteor API Docs

@rjdavid thank for your reply, please example I’m not clear.

I have 2 methods in my Utility class for form events.

/**
   * Have to call enableElement after calling disableElement
   * Because disableElement will lock the cursor on body element
   * @locus client
   * @param element {string}
   * @param text {string}
   */
  static disableElement(element, text = 'Loading') {
    $('body, button, .btn, a, input, select, textarea').css('pointer-events', 'none');
    $(':button').prop('disabled', true); // Disable all the buttons

    if (text !== '') {
      text = `${lang(text)}...`;
    }

    $(element)
    .data('html', $(element).html())
    .html(`<i class="fa fa-spin fa-spinner"></i> ${text}`)
    .prop('disabled', true);
  }

  /**
   * Enable element
   * @locus client
   * @param element {string}
   */
  static enableElement(element) {
    window.setTimeout(function () {
      $(element)
      .html($(element).data('html'))
      .prop('disabled', false);

      $(':button').prop('disabled', false); // Disable all the buttons
      $('body, button, .btn, a, input, select, textarea').css('pointer-events', 'auto');
    }, 200);
  }

After a click I’m disabling button and enable it in callback. Like

'submit form#update-payment-types'(e, t) {
    e.preventDefault();

    const button = 'form#update-payment-types button[type="submit"]';
    FormUtility.disableElement(button);

    ordersCustomersUpdatePendingOrders.call({
      storeId: Profile.storeId,
      customerId: FlowRouter.getParam('id'),
      paymentType: e.target.type.value
    }, (error) => {
      FormUtility.enableElement(button);

      if (error) {
        log.error(error);
        return;
      }

      t.state.set(STATE.PENDING_ORDER_COUNT, 0);
      t.state.set(STATE.PENDING_ORDER_TOTAL, 0);
    });
  }

@minhna On client when submit it loading whole form and on server i use rate limit but user insert transaction it is still insert duplicate records again.

@guncebektas On client when submit it loading whole form and on server i use rate limit but user insert transaction it is still insert duplicate records again.

1 Like

Can you please share your form and other related server side code?

How do you use it? what’s interval value? You should share some of your codes so other people can help. Don’t worry, we won’t steal your code.

@minhna @guncebektas Sorry for late replying and this is my code on server

import rateLimit from '/imports/api/lib/rate-limit'

export const insertSale = new ValidatedMethod({
  name: 'pos.insertSale',
  mixins: [CallPromiseMixin],
  validate: inUpSchema.validator(),
  async run({ doc, itemDetails, opts }) {
    if (Meteor.isServer) {
      /*** Sale */
      let saleId, tranType
      try {
        const saleDoc = new Sale(doc)
        const res = await saleDoc.save(opts)
        saleId = res.id

        // hook មាន auto generate refNo
        doc.refNo = res.refNo
        doc._id = res.id
        tranType = doc.tranType

        const logData = cloneDeep(doc)
        // Pick data
        const {
          saleDetailsData,
          saleOrderDetailsData,
          inventoriesData,
          // inventoryIds,
          cashWithdrawalData,
          journalData,
          journalDetailsData,
          posInventory,
          posRevenue,
          // errorOnHand
          template,
        } = await pickSaleData({
          saleId,
          refNo: doc.refNo,
          tranDate: doc.tranDate,
          customerId: doc.customerId,
          employeeId: doc.employeeId,
          tranType: doc.tranType,
          warehouseId: doc.warehouseId,
          chartAccountIds: doc.chartAccountId,
          branchId: doc.branchId,
          memo: doc.memo,
          // cashChange: doc.cashChange,
          // Details
          saleDiscount: doc.totalDiscount, // Feature
          withdrawalAmount: doc.withdrawal,
          remaining: doc.remaining,
          itemDetails: itemDetails,
          endDate: opts.tranDate,
        })

        /*** Sale detail */
        await SaleDetails.insertMany(saleDetailsData)
        /*** Sale order detail */
        if (saleOrderDetailsData.length) {
          await SaleOrderDetails.insertMany(saleOrderDetailsData)
          await updateSaleOrderStatus({ refSaleOrderId: doc.refSaleOrderId })
        }
        /*** Deposit */
        if (cashWithdrawalData) {
          await insertCustomerDeposit.call({ doc: cashWithdrawalData })
        }

        /*** Inventory */
        if (inventoriesData.length) {
          await Inventories.insertMany(inventoriesData)
        }

        /*** Journal entry */
        if (journalDetailsData.length)
          await insertJournal.call({
            doc: journalData,
            details: journalDetailsData,
            posInventory,
            posRevenue,
          })

        // Audit log
        logData.details = saleDetailsData
        await auditLog.add({
          page: `Sale-${doc.tranType}`,
          title: logData.refNo,
          data: EJSON.stringify(logData),
          refId: saleId,
        })

        // Remove Hold Sale and Hold Sale Detail
        if (opts && opts.holdSaleId) {
          const preHoldSale = await HoldSale.findOne({ _id: opts.holdSaleId })
          // if (!preHoldSale) throw `This transaction deleted!`
          if (preHoldSale) {
            // remove hold sales
            await preHoldSale.remove()
            // remove hold sales details
            await HoldSaleDetails.deleteMany({ saleId: opts.holdSaleId })
          }
        }
        // send to telegram
        sendTelegramBot({ template, branchId: doc.branchId, tranType: 'Sale' })

        return saleId
      } catch (error) {
        console.log(`${doc.tranType} InsertError`, error)
        if (saleId) {
          // pre doc of sale details
          const prevDoc = await SaleDetails.find({ saleId }).lean()
          const ids = prevDoc.map((it) => it._id)
          // Remove inventory
          await Inventories.deleteMany({
            refId: { $in: ids },
            refType: tranType,
          })
          // Journal
          await removeJournal.call({ refId: saleId, journalType: doc.tranType })
          // Sale order
          await SaleOrderDetails.deleteMany({
            refId: saleId,
            refType: tranType,
          })
          await updateSaleOrderStatus({ refSaleOrderId: doc.refSaleOrderId })
          // Sale
          await SaleDetails.deleteMany({ saleId })
          await Sale.deleteOne({ _id: saleId })
          const data = {
            error: new Meteor.Error(error),
            doc,
            itemDetails,
          }
          auditLogError({
            page: `Sale-${doc.tranType}`,
            title: `${tranType}-${saleId}`,
            data: EJSON.stringify(data),
            refId: saleId,
          })
        }
        throwError(`Insert ${doc.tranType} Error`, error, { doc, itemDetails })
      }
    }
  },
})
rateLimit({  
  methods: [insertSale],
  limit: 1,
  timeRange: 1500
})

Any suggestion

I saw you don’t check for duplicates in your method. It’s all depends on rateLimit function. How does it look? have you test it? does it work?

Not sure how Mongo’s eventual consistency will handle this under load, but, you can always have the client store a unique ID (perhaps generated by the server and retrieved by a Meteor method) that kind of acts like a request token / ticket in a queue. That way if you see duplicate attempts to insert a document at that ID, you can figure there’s either an error or you need to update existing data, rather than duplicate it.

At least I’m pretty sure that’s how old school forms used to work before a lot started to be hidden from user view and JS took over so much. And you’ll often see in the browser the classic “are you sure you want to resubmit this form?” when going back through your browser history far enough. Basically you need to reimplement that browser level protection (which users URLs + post requests I guess as a unique identifier) in JS. And yes it could be abused… always good to check ownership of documents + user IDs to make sure everything lines up…

Probably some parallels to a CSRF token, but instead of cross-site request forgery what you’re trying to do is prevent duplicate requests without the user intending to do that.

@minhna Rate limit allow only avoid from request for client. What should I check for duplicate?

@ceigey I generate id auto so it don’t duplicate but it duplicate document that insert