I have an interesting problem where I have a method call to the server which kicks off a series of async requests. These requests progressively collect a large amount of records (300k+ sometimes) from different APIs and then post the aggregated list to a final API. However, sometimes, if the number of requests i’m making gets too large, Node runs out of memory because that memory can not be garbage collected till all the async requests are resolved and the app crashes with the “<— Last few GCs —>” memory error. (this is expected behavior, i looked it up).
Here’s the problem: Because the initial method call comes from the client, if the user is still connected that method will start over again on the server when the app boots back up. So, i end up with repeated crashes that seem to be automatically restarting.
I think this has to do with the process getting interrupted, but the method still being in the pipeline and never finished. I think this is actually a feature of Meteor’s that’s ended up hurting me a bit in this instance.
My question is… is there a way to clear out the Meteor queue on crash or restart? Has anybody else experienced something like this? thanks!
This is because the client retries the request when the connection comes back up. You could pass the current time in the original call. If the server’s method sees this is more than x seconds old (because it’s being retried much later), it could just return an error. Clearly, you should allow for some leeway due to client and server time not being perfectly in sync.
Oh, i see. this would explain some other behavior. So, if the connection gets broken before the call is completed, the client re-tries it. I didn’t realize that was the process. i thought it was sent, queued on the server, and the client just checked back in. I was also seeing multiple instances of a call happen as well, which would be explained by this behavior. sometimes this particular function can take several minutes.
Passing in a date/time is a good idea. I might also try returning something, so as to complete the client side request, and allowing the other methods to operate asynchronously in the background.
There is a setting on Meteor Validated Methods (maybe the regular methods also)
I believe it disables the functionality of retrying automatically
Other options you might be interested in passing are:
noRetry: true This will stop the method from retrying if your client disconnects and reconnects.
// This is optional, but you can use this to pass options into Meteor.apply every
// time this method is called. This can be used, for instance, to ask meteor not
// to retry this method if it fails.
applyOptions: {
noRetry: true,
},