We are currently scaling our meteor app. Currently, we are on a monolithic architecture with Meteor on Galaxy & MongoDB on Atlas. We are planning to scale up & only some modules of the app will require higher CPU, Storage & RAM.
We are wondering if it makes sense to go microservices or stay monolithic with autoscaling on? Appreciate your advise.
Thanks
Jay Thattai
I’d very strongly advise microservices here - depending on your exact requirements. I’m not a fan of “mesh” architectures, but if your meteor applications are going to be driving these requests, 100% you’re going to want separate containers for your high CPU functionality - the key reason being that it won’t impact the responsiveness of your application
Though I agree, its also worth considering the costs of having a microservice architecture. I know from experience that the complexity increases exponentially. Sometimes it just makes more sense to just throw a few extra bucks against it in stead of spending this money on an increasing dev cost from which I know might be even more expensive in a lot of cases
In the end its a matter of which part needs scaling. It could be that just the backend part is split. In that case its not that complex
@cxoncloud Would you please care to list some of the things you thing you might want to externalize.
For my case:
A concept of microservicing is also to push processes as much as possible to users. For example: mapping arrays, destructuring, data validations, uploads of files to CDNs, image processing (canvas resizing etc).
That’s something I hear too, that micro-services can become complex and expensive.
In our case we have 8-10 key functionalities like set up, bookings, feedback, 3rd party integrations, marketing, UI & UX, analytics…
Our road map is to create an API set for 3rd parties to consume any service of ours. Our simple thought process was to create these functionalities as individual services that can be exposed as an API.
We don’t do any image or video processing. We might do some web-scraping in the future but nothing heavy that I can think off.
Note that if you deliver your app through a mobile build (android/ios) via Cordova you can also try to outsource things like image/video processing to your clients.
My preferred setup with Meteor in a micro service setup is to have Meteor take care of just the frontend (as a BFF and App). This means that any backend data is collected from one or more services into MongoDB where Meteor just does its default thing. Mongo essentially acts as a read database.
It is based on my favorite architectural pattern CQRS (Command Query Responsibility Segregation) where the Meteor methods do API POST, PUT whatever requests on external (micro) services. Then a fetch is done to retrieve updated data which is stored into MongoDB into the exact format that is needed to read it.
This prevents unwanted complexity on the Meteor side and lets Meteor do what its best at - serving frontends. But the responsibility of maintaining specific chunks of data lie on the micro-service side.
Everything can become complex and expensive – especially monolithic systems. It is not without a reason that monolithic systems are susceptible to evolve towards the anti-pattern called Big Ball of Mud. Microservices were invented in the first place to avoid going down that path.
This is not to say that a microservices architecture comes with a guarantee that the final system is going to be nice and clean. It depends 
I use microservices in my project, and so far everything went well. Components are primarily interconnected using Apache Kafka as a message broker; they receive each their input from a specific topic and write their output (mostly) into another topic, consumed by yet another component unless circumstances dictate otherwise: some components write straight into MongoDB.
Our Meteor app initiates most of the microservice actions by sending a command via one of the Kafka topics; in other cases, cron starts a script that sends a command via Kafka. In some cases, the microservice uses a MongoDB change stream as a trigger, and sometimes components send each other commands.
Most of our components are implemented in node.js, some others in Java. The beauty of this is the infinite scalability: the point isn’t just that the Meteor server is reduced to the essentials, it’s also that the tasks are scaled out to any number of servers by the mere virtue of how Kafka works. The core idea is here that Kafka topics can be partitioned, and thus multiple consumers (services) can jointly process the messages if scalability is needed.
I encourage everyone to consider a similar architecture.
To me, the real eye opener was the article The Log: What every software engineer should know about real-time data’s unifying abstraction by Jay Kreps in 2013, back then the CTO at LinkedIn if I’m not mistaken. Highly recommended reading!
I understand why people dont like monolythic apps. Mostly they are built wrongly with a lot of spagetti code and with a lot of cross dependencies.
For that I’ve introduced a term that is badically just a best practish, but I found it necesarry to name it. Decoupled composition apps. Decoupled composition apps are applications that are simply composed out of specialized NPM packages or at least modular folders that act as a standalone part of your app (no cross deps to other modules).
An example of such a package would be the design system implementation. It contains purely UI and components and dictates the entire design of your app. This way your app does not have or barely has any styles of its own. The complexity of the design is now in the package and it could be reused in other apps.
You could do the same with the API side where you build everything decoupled. You simply compose stuff into your app and if you have a situation where load needs to be handled differently, you could simply externalize or cut/paste that part into its own project.
Love this pattern. More people should be aqare of this 
You’re absolutely right: with the right approach taken, applications of any size can stay maintainable, including very large ones. Nothing forces us to create a big ball of mud; in fact, the intention behind most of the software design principles is to avoid it.
So if someone contemplates introducing a microservice architecture, tidiness and clean design should not be the main motivators, for a large single application can (and should) also be tidy and clean, with clear separation of concerns, with layers, responsibilities, objects, domains and whatnot.
In fact, if someone feels that they can’t develop and maintain a large yet tightly organized application, they should certainly keep clear of microservices, because they usually come with an elevated complexity. Even if nothing else, deployment can grow far more complex.
To me, the main motivation for a microservice architecture are the separation of deployment and scalability. These concerns rarely ever play a role with small applications.
Well said! Wish more people would realize this!