How to change a meteor method to an independent thread

[truran]

0 down vote favorite

I am using meteor.js. I want to call a thread that is independent of the thread that called it. In other words the spawned thread is unattended and does not return a value to the thread that spawned it. The spawned thread will update the database and meteor will take care of updating the client reactively

The reason I want independent/unattended threads is that I want my application to scale vertically. Imagine a situation where a million plus users are calling updateCompanyAmount (which updates a shared database collection’s document and value) at the same time!! Some users connections might timeout while waiting for the lock to be released so they can update the shared value.

Here is my code. Please read the four comments in the code to quickly understand the code. Please give me a solution in javascript. Can an async call solve this issue? If so, why, and can you give me an example of meteor async method.

if (Meteor.isServer) {
	Meteor.methods({
		//I want to "save purchase history" and then return to client while a new thread or service-worker "updates company amount"
		savePurchaseHistory: function(recharge,var1,var2,var3,var4,var5)
		{
			var userId = Meteor.userId();
			.
			.
			.
			PurchaseHistory.insert(recharge);
			Meteor.call("updateCompanyAmount",companyAmount);
		},
		//I want to change this function to a service-worker or Meteor microservise
		updateCompanyAmount: function(companyAmount,balance) {
			Meteor.call("updateCompanyAmountRecursively",companyAmount);
		},
		//CompanySharedNumbers collection can be updated by multiple users. In this example companyAmount in CompanySharedNumbers is being updated
		//by multiple users using a lock!
		updateCompanyAmountRecursively: function(companyAmount) {
			var userId = Meteor.userId();
			
			var user = Meteor.users.findOne({_id: userId});
			var random = Math.random();
			var companySharedNumbers = CompanySharedNumbers.findOne({_id: 'onlyOccurrence'});
			var locked = companySharedNumbers.updateLock;
			if (!locked)
			{
				CompanySharedNumbers.update({_id: 'onlyOccurrence',updateLock:locked},{$set:{updateRandomNumber:random,
						updateLock:!locked,updatedBy:user.username}});
				Meteor.users.update({_id: userId},{$set:{updateRandomNumber: random}});
			}
			
			user = Meteor.users.findOne({_id: userId});
			var userRandomNumber = user.updateRandomNumber;
			companySharedNumbers = CompanySharedNumbers.findOne({_id: 'onlyOccurrence'});
			var randomNumber = companySharedNumbers.updateRandomNumber;
			var updatedBy = companySharedNumbers.updatedBy;
			locked = companySharedNumbers.updateLock;
			if (locked)
			{
				if (randomNumber == userRandomNumber && updatedBy == user.username)
				{
					if (companySharedNumbers.companyAmount)
					{
						companyAmount = companyAmount + companySharedNumbers.companyAmount;
					}
					
					random = 2;
					CompanySharedNumbers.update({_id: 'onlyOccurrence',updateRandomNumber:userRandomNumber,updateLock:locked,updatedBy:user.username},{$set:
							{updateRandomNumber:random,updateLock:!locked,companyAmount:companyAmount,updatedBy:user.username}});
				}
				else
				{
					Meteor.call("updateCompanyAmountRecursively",companyAmount);
					return false;
				}
			}
			else
			{
				Meteor.call("updateCompanyAmountRecursively",companyAmount);
				return false;
			}
		}
	});
}

[cloudspider]

This might be a recommended read for your use case:

https://medium.com/@pmuens/meteor-defer-and-this-unblock-e610c8426065

[robfallows]

Also this:

Fun with Meteor Methods – Meteor Blog

Also, it’s probably worth mentioning that (until very recently) threads in NodeJS weren’t really a thing (NodeJS is single-threaded), so if you really did need a separate thread you’d spawn another process.

[truran]

Thank you. I’ll read and update.

[truran]

@ cloudspider, in the example, what happens if the user closes the browser after signUp before Meteor.defer(function () {Email.send(/* email object */);}) is executed. Will the email sending happen or will it not?

[cloudspider]

I think that if you wrap it in a defer, it will not stop it. I’m not completely sure though. Will try this tomorrow morning.

[truran]

Thanks. Your help in this regard is and will be much appreciated.

[truran]

With Meteor.defer my recursive function always stops at “recursive function loop” 610 for some reason. I tried @robfallows async function and I close the browser the recursive function continues to run in the background as per my use case requires. I even looped to 10000, no problem. The only problem is that

var userId = Meteor.userId();

returns ‘undefined’ inside the async function!! Is there a way to get the userId inside the async function?

My changed code:

if (Meteor.isServer) {
	Meteor.methods({
		savePurchaseHistory: function(recharge,var1,var2,var3,var4,var5)
		{
			var userId = Meteor.userId();
			.
			.
			.
			PurchaseHistory.insert(recharge);
			Meteor.call("updateCompanyAmount",companyAmount);
		},
		updateCompanyAmount: function(companyAmount,balance) {
			var count = 0;
			Meteor.call("updateCompanyAmountRecursively",companyAmount,count);
		},
		async updateCompanyAmountRecursively(companyAmount,count) {
			//var userId = Meteor.userId();
			
			.
			.
			.
			locked = true
			
			if (count == 10000)
			{
				if (companySharedNumbers.companyAmount)
				{
					companyAmount = companyAmount + companySharedNumbers.companyAmount;
				}
				
				random = 2;
				CompanySharedNumbers.update({_id: 'onlyOccurrence',updateRandomNumber:userRandomNumber,updateLock:locked,updatedBy:user.username},{$set:
						{updateRandomNumber:random,updateLock:!locked,companyAmount:companyAmount,updatedBy:user.username}});
			}
			else
			{
                                count =  count + 1;
				Meteor.call("updateCompanyAmountRecursively",companyAmount,count);
				return false;
			}
		}
	});
}

[truran]

The second problem is that when I call the async function the spinner continues spinning on the browser. The next page is not displayed.

[truran]

When I call the async function from the client both problems are solved.

[cloudspider]

Actually to preserve the userId, its the same as any other javascript context. Consider this:

for(let i=1; i<=10; i++) {
  setTimeout(() => {
    console.log(i);
  }, 1000);
}

In the above example it would show 10 times the number 10! That is because of the setTimeout function is only triggered after the loop has finished which means that i was updated already to 10.

To fix this you would need to push the variable to the timeout function:

for(let i=1; i<=10; i++) {
  setTimeout((currentNumber) => {
    console.log(currentNumber);
  }, 1000, i);
}

Alternatively you could do:

for(let i=1; i<=10; i++) {
  const currentNumber = i;
  setTimeout(() => {
    console.log(currentNumber);
  }, 1000);
}

For Meteor.defer this works the same:

Meteor.methods({
  this.unblock(); // Allow client to do other method calls too

  const userId = this.userId; // Notice I'm using this.userId instead of Meteor.userId() since this.userId comes from the server while Meteor.userId comes from the client 

  for(let i=1; i<=10; i++) {
    Meteor.defer(() => {
      Meteor.call('someLengthyOperation', userId);      
    });   
  }
});

[truran]

@ robfallows and @ cloudspider thanks for your help. Much appreciated.

[hazemkhaled]

Using Microservices with Meteor helped me a lot, This is step by step how to run a specific method or publication in anither process

https://link.medium.com/dMw8HMTDwS

[magallaher]

Thanks for the case. Found some useful info.