I too was in a very similar situation. I needed to parse bulks of large uploaded files, store them in Mongo, the process the imported data. Since these are relatively large data sets, I was getting Mongo concurrency issues and timeout issues as well, not to mention processing a few hundred thousand rows was taking hours to complete. I found this since I pay for Bullet Proof Meteor (totally work the price btw).
I copied @arunoda’s worker code and created an ES6 worker class, then genericized it a bit so I just have one method to call (processWorker) to the Worker class to process each separate Meteor Method. Then I separated my Meteor methods into a local package called “my-workers”, and created a separate Meteor project that simply starts Meteor and exposes those packages. Now I can run N number of the worker Meteor projects on separate ports (or even separate machines) and simply call the Meteor method through the worker class instance (which are all pointed at a single Mongo container).
The solution took that several hour processing time into minutes (and even seconds in some cases). It is not a glamorous solution but it is REALLY lightweight, simple, and easy to implement. Right now the only pain point I have experienced is the configuration of worker URL’s is manual but it looks like the meteorhacks:cluster package will solve this.
Let me know if you want more details I am happy to share.
1 Like
This sounds hugely helpful and will attempt to implement it! 
Depending on your data you might be able to use the MongoDB aggregation framework to offload the work to the DB.
1 Like
You can use a package I developed that may be shipped with 2.0 natively : nschwarz:cluster.
It’s a worker pool that enables you to run jobs on workers (if your cpu is multi-core).
1 Like