I’m working on app which, at the moment, contains a few separate parts:
A desktop chat app built with Electron.js (think Slack).
A simple node.js server that the desktop app communicates with. The desktop app uses socket.io to send messages to this server, which in turn emits the message to all connected clients.
A newly created Meteor.js app, which at the moment is acting as the download page for the desktop app, and a few other functions (like payments).
I have a few questions:
I’m thinking about scrapping the node.js app and implementing its socket.io functionality into the Meteor app. Some concerns include scalability. Will having a single Meteor.js app serve the web pages and receiving a bunch of socket.io messages cause scale issues? Is it better to keep my current architecture with a seperate node.js app for the chat system?
I need to implement user authentication in the Electron app so users can log in. If I decide to scrap the node.js app as mentioned above and just use the Meteor.js app, how can I connect Meteor’s default user system to the Electron app to allow users to login?
I myself don’t have much experience in the problem you stating but I’m sure some people here do. If you allow me to suggest shortening and narrowing down your problem statement to a specific technical question which seems to revolve around the point below then you might have a better luck
Let’s start with few questions that can lead us to better answer.
Ultimately, how many users are supposed to use this app. Is that for internal use among some company workers? How many chatrooms are planned? I know you consider scaling, but this would still be useful to be addressed.
Will the messages ever be saved to the database,or only passed further to all available clients at the moment? Will users or admins have some access to chat history, will there be some way to mark messages as unappropriate (spam, abuse, other potentially unwanted stuff).
That said, by default, Meteor’s pub/sub system is not the best option for large scale chatting applications. This may be better done with some simple system like Socket.io or Deepstream, unless you use some customized system to send messages with Meteor, f.e. as streams. New Redis Oplog comes pretty handy with that, but as it’s quite new, there’s not enough data yet to prove it at large scale.
If your main reason to choose Meteor is because of the authentication system available out of the box, I’d reconsider it. The authentication in Meteor is awesome, I admit, but that shouldn’t be your main worry.
There is no predefined number of users. I’d like to be able to scale as much as possible (within reason) without running into issues, whether that be 10 users or a thousand.
While not implemented yet, yes, messages will be saved to a database. This is so that when new members connect, I can retrieve a history of messages.
As to your final comment, I am using socket.io. The current desktop app connects to a barebones node server and communicates via socket.io, and currently does not use Meteor in any way.
I haven’t worked with Electron but I don’t imagine there would be any problems with using Meteor’s user authentication system for your use case.
The trick is that the way to scale your system with Meteor way would be by… not using what Meteor is best at. That means not using the data-sync feature allowing other users to sync their client state with the database, but to get the new messages from streams ommiting the reactive database writes and reads (making the writes non-reactive).
There are few Meteor-based chat applications that do it this way more or less, you can check f.e. Rocket Chat.
Yea, as mentioned the desktop chat app isn’t using Meteor at all. It’s just a plain Electron/Node/React app.
As per user authentication, how would I do that though? I have been unable to find any relevant documentation. It’s not like I can call Meteor.loginWithPassword() from the Electron app.
Sorry, I had wrong impression that you would be rewriting your front-end too.
Then, the answer is - personally I wouldn’t bother with such a structure. It’s possible to achieve but not worth it imho.
You will spend way more time rewriting stuff and forcing it to scale than you would to just adopt some 3rd party authentication system.
That said, if you want to make it work, you will need a DDP-client for your front-end app that works with DDP-based authentication. There are libraries on Github that achieve that, but they’re old and most probably outdated.
I’ve not been in situation where I’ve thousands of concurrent users chatting simultaneously so take my opinion with a grain of salt, but I don’t think the expectation of scaling indefinitely without careful engineering at the platform level and not experiencing any performance issues is a realistic one, there are reasons why those platform as as service exist.
I’m really just interested to know if Meteor will have a problem performing the duties of both the current node.js app (that is, handling a bunch of socket.io messages) and serving a web application. Obviously, being able to just have one server side app and the one desktop app to manage would be easier than having 3 different components to manage.
If you know ahead of time that certain parts of your app are going to draw on resources heavily, I’d build anticipating a move to a microservice.
For instance, we listen for mailgun notifications. If users are batch sending, then these can often hit several thousand requests in just a second or two, typically enough to crash a galaxy container, or at least grind things to a halt. That’s not something I’d like to scale with galaxy. So we have some super simple node scripts running on DO servers to handle those requests, and scaling those up is much more cost effective.
That means not using the data-sync feature allowing other users to sync their client state with the database, but to get the new messages from streams ommiting the reactive database writes and reads (making the writes non-reactive).