Upload using meteor method slow

help
1

Hello,

I’m building a program that allows people to upload a CSV, which will be processed on the server. I got it working using reader.readAsText, and then dumping the contents into a meteor method call. But when I decided to stress test it, I made a CSV 100k lines long (which my program needs to support).

It took minutes to upload locally, and my CPU usage was very high for meteor tool. I’m pretty sure the problem is that Meteor is trying to run EJSON.parse on the csv file. Is there a way to disable it?

Then I tried using webapp.connecthandlers, but realized there is no way to get Meteor.user() in this, which I need, so I am not sure what to do. If anybody has any guidance, that would be appreciated.

WebApp.connectHandlers.use(`/email-list/process-email-csv`, function (req, res, next) {
	try {
		var Fiber = Npm.require('fibers');
		Fiber(function (self) {
                 //begin processing
		})()
	} catch (e) {
		res.writeHead(500);
		res.end(e);
	}
});
2

Actually I think I may have found the culprit :grin:

	items = _.uniq(items, function (item, key, email) {
		return item.email;
	})

Whoops! That sure takes a while on 100k records! I found a better way to ensure uniqueness.