My experience is that it’s higher. 5-10% CPU on a single $10 server with 250 connections. I suspect the “long runningness” may be some part of it - a connection that isn’t consuming any data really has no CPU cost (though it often does have a memory cost).
It’s not easy to give advice without knowing a lot about the type of data (read/write volume and computation needs mostly). If your meteor server is basically acting as a passthrough from DB -> client and there are limited reads, then scaling will be relatively easy.
Choosing whether to spin up a container/DB for each customer (single tenant) or keeping them in one is usually more a security consideration - either one can work, and either one can be scaled. Each has it’s own drawbacks (poor load balancing for intermittent use customers vs ability to tell a customer “it’s slow cos you’re using it a lot”).
I personally would never run the DB on the same host as the app server - I prefer a separation of concerns (for performance and security) and I’d always run a full replicaset rather than a standalone (for resiliency). But depending on the type of service you’re selling, your requirements may be different.
@znewsham, regarding db on the same host - I understand your point about resiliency, but like you say that depends on the type of service you’re selling.
But regarding performance - provided neither the meteor process nor the db are processor hogs you can’t get more performant than db on the same host with zero latency. But yes, when one starts slowing down the other then they need to be separated.
Regarding security - My db is closed to the outside world. I don’t have to expose any ports on my host just for the db because the app can access it on localhost, so also more secure, no?
This works only as long as you don’t need to scale out with your app. As soon as you do, you’ll have to expose the db’s port anyway. But even that is not a security problem if you set up a firewall (e.g. iptables) that makes the port inaccessible again – except for the dedicated ip numbers of where your meteor instances (and, alternatively, mongodb replica set members) are.
What @znewsham may have meant with security is that if you have multiple services on the same machine and one of them has such a security flaw that allows for the execution of an arbitrary code by the attacker, then the data from the other service (e.g. a complete mongodb database) could be copied.
Galaxy is not a bad option, specially they give you monitoring, security is another thing to consider.
Digital ocean recently started offering “app platform” not really sure if it’s any good, but it’s advertised that nodejs + mongodb can be hosted in single app for around $12 or so.
Paradoxically, the more and the better such services pop up, the more is Meteor’s existence in jeopardy, meaning that Meteor’s increasing popularity can also seal its own fate – not technologically, but in economic terms. MDG abandoned Meteor for unknown reasons, and it’s not far fetched to assume that their business model didn’t pay off. It was based on Galaxy, and obviously, that’s the same with Tiny.
Apparently, the team was much larger with MDG, i.e. their monthly recurring costs must have also been much higher, but Tiny still has the very same problem: If they don’t get enough customers for Galaxy sustainably, they will go down. Also, there’s an investor behind Tiny, and they want to see a profit.
From this perspective, MUP / MontiAPM and all the app platforms are parts of a problem for Tiny, and, from a broader perspective, a problem for all whose product is based on Meteor.
@wildhart Thank you so much! I will definitely keep these in mind.
I think my main issue currently is that I am making too many calls to the database because I couldn’t figure out how to get all my data in one call.
I am using Meteor & React. I have a collection that can be a parent of itself. I couldn’t figure out how to get all connecting children and sub children etc from one call. It’s explained in more detail here.
I essentially currently render one HOC using withQuery and then an unlimited number of child HOC withQuery components within each child (if that makes sense?)
I guess I will do my best to optimise performance prior to launch and then work from there.
@znewsham - My initial thought was to separate the database, which I will likely end up doing. Maybe not at initial launch, but soon after I suspect.
@peterfkruger - Thank you for helping (again)! Is MUP the best option for deployment? For some reason, I had been under the impression that it was not supported anymore.
@deank - Galaxy seems to have good tools but the pricing is high. I also recently saw the DO App Platform but I was slightly confused but it’s pricing. It seemed that you signup for the app platform, but also then pay fees on top of that for hosting?
That’s why Tiny is improving Galaxy. They have to. That’s how they win. Meteor is open-source so they know there’s going to be many solutions for self-hosting, etc. They have to make Galaxy the easiest, most feature-rich, most economical, solution. Galaxy could use a lot of work on the economical side of things.
I can’t tell you if it’s the best option. I don’t use it at the moment, but I will probably give it a try, since it is looking good and has just about the features that I need.
In plain language, Galaxy needs to be a lot cheaper; and then probably a lot more people would not bother anymore with custom deployments, they would much rather go straight to Galaxy and stay there.
I believe it was a sound idea back then with Meteor’s previous owner to try to sell Galaxy’s service at a high price. But it already should have been realized since that the profit curve maximum is not nearly at that price.
Have you tried mongo aggregation with $graphLookup? I’ve never used it myself, but I use aggregation in other ways.
The MUP repo has been updated regularly, most recently 4 months ago. It has a growing list of plugins which have been added relatively recently. The author @zodern is active on these forums and is also the developer of MontiAPM.
Having said that, I would love to support Tiny if Galaxy was cheaper.
I’m starting to use Docker in a simple server. I think that way is better to future increase of performance. You can start with a simple Docker or a Swarm with only one node. And if you App comes to have a lot of connection, you will be prepared to expand your structure.
To host, I’m using UpCloud. Little more expensive that DO and others, but the hardware is so much better. You can use my referral link to receive a bonus of $25 . https://upcloud.com/signup/?promo=28S7CS
You can use MUP too, is an easy way to deploy in your own server. And you don’t need to know about containers or others infrastructure things.
@raragao Does running a Docker container improve performance over running directly on a server / VPS?
What do you mean by “better” hardware? From what I’ve seen, the pricing between Lightsail/Linode/DO/UpCloud/etc. is fairly similar for the same spec of hardware (memory, cpu, storage, transfer)? Is there something I’m missing?
Docker doesn’t improve performance, but allow you scaling vertical and horizontally your system when you need. With VPS or a VM, you can’t or you need a lot of work to do (and slow work). This is very simple when you use a Swarm, per example.
When I told better hardware I mean about quality and performance of hardware, not sizing. I want to tell that performance of hardware of UpCloud is better than others cloud with same size. I did this comparation with same sizing (RAM, CPU and I/O).
I can test it and compare with others (DO, Heroku, AWS, etc).
@wildhart. Hmm, I must’ve got confused somewhere. I will look into using MUP for my deployments.
I have not tried graphLookup aggregation, but I will try to use that instead. I currently use Grapher React for pub/sub and createQuery so I will try to implement aggregation in there - thanks for the suggestion!
I guess it may be true that with a VM / VPS scaling up / out can be difficult or even impossible; but if that’s the case, the reason is always incompetence, or, to put it in a nicer way, lack of expertise.
To make it perfectly clear: If you have the required knowledge, scaling up or scaling out a Meteor app in a VM / VPS environment should not be difficult at all.
A final word about VPS: don’t go there, it’s a bad choice. In case of a VPS you’re getting just nominal resources in terms of number and power of CPUs, amount of RAM and network throughput / latency, because that’s how a VPS works: you’re on a hardware shared with other clients, and you have zero guarantee over what those other folks do and how many they are. It is of course perfectly ok if your app has zero or just marginal importance, maybe because it’s just for a quick demo or a proof of concept or similar. It is a bad idea to use a VPS with a real-life app, especially if it’s commercially relevant.
A VM is like a VPS, except that all resources are dedicated to you, nothing is shared with any other client. AWS EC2 instances are for example real VMs, as are those that one can commission for themselves on a rented physical root server.
Yes, you can use a container for each tenant/client, or more than one container (scaled horizontally) when you need more throughput.
Container by concept should to run one unique service. To databases, create other container.
Yes, swarm can distribute traffic to specific container, but to web app or service, my advice is use some container with a reverse proxy (Apache, Nginx, Traefik).
Really, you have many options. You need to know if you have need to do every manually or use some IaaS or PaaS to host your apps. I already use Galaxy, Scalingo and Nodechef. For now I create my own cluster in UpCloud IaaS.