Oh, in the meanwhile you rewrote the question. So I’ll prepare second answer.
-
There is no predefined number of users. I’d like to be able to scale as much as possible (within reason) without running into issues, whether that be 10 users or a thousand.
-
While not implemented yet, yes, messages will be saved to a database. This is so that when new members connect, I can retrieve a history of messages.
As to your final comment, I am using socket.io. The current desktop app connects to a barebones node server and communicates via socket.io, and currently does not use Meteor in any way.
If I understood you correctly then basically your questions are:
- How many concurrent chat users Meteor oplog can handle without issues
- How to integrate 3rd party authentication to Meteor accounts system
Is that correct?
I haven’t worked with Electron but I don’t imagine there would be any problems with using Meteor’s user authentication system for your use case.
The trick is that the way to scale your system with Meteor way would be by… not using what Meteor is best at. 
That means not using the data-sync feature allowing other users to sync their client state with the database, but to get the new messages from streams ommiting the reactive database writes and reads (making the writes non-reactive).
There are few Meteor-based chat applications that do it this way more or less, you can check f.e. Rocket Chat.
2 Likes
I’m not familiar with Meteor’s MongoDB Oplog system but after a quick read I’m gonna say yes.
Also, yes, how to authenticate users in my Electron app using the accounts system from a Meteor backend.
Yea, as mentioned the desktop chat app isn’t using Meteor at all. It’s just a plain Electron/Node/React app.
As per user authentication, how would I do that though? I have been unable to find any relevant documentation. It’s not like I can call Meteor.loginWithPassword() from the Electron app.
Sorry, I had wrong impression that you would be rewriting your front-end too.
Then, the answer is - personally I wouldn’t bother with such a structure. It’s possible to achieve but not worth it imho.
You will spend way more time rewriting stuff and forcing it to scale than you would to just adopt some 3rd party authentication system.
That said, if you want to make it work, you will need a DDP-client for your front-end app that works with DDP-based authentication. There are libraries on Github that achieve that, but they’re old and most probably outdated.
To confirm, when you say
I wouldn’t bother with such a structure
what structure are you referring to?
I am fairly against using third party auth systems. They are usually more trouble than they are worth, especially for such a critical part of an app.
Javascript client not being a Meteor app, with Meteor backend, using Meteor auth system to write a chat application.
You also mentioned:
You will spend way more time rewriting stuff and forcing it to scale than you would to just adopt some 3rd party authentication system.
Do you have any examples of what kind of scale issues?
I’ve not been in situation where I’ve thousands of concurrent users chatting simultaneously so take my opinion with a grain of salt, but I don’t think the expectation of scaling indefinitely without careful engineering at the platform level and not experiencing any performance issues is a realistic one, there are reasons why those platform as as service exist.
I also tend to agree with gusto statement above.
Which statement is that?
Apologize for the ambiguity, the following statement:
I’m really just interested to know if Meteor will have a problem performing the duties of both the current node.js app (that is, handling a bunch of socket.io messages) and serving a web application. Obviously, being able to just have one server side app and the one desktop app to manage would be easier than having 3 different components to manage.
If you know ahead of time that certain parts of your app are going to draw on resources heavily, I’d build anticipating a move to a microservice.
For instance, we listen for mailgun notifications. If users are batch sending, then these can often hit several thousand requests in just a second or two, typically enough to crash a galaxy container, or at least grind things to a halt. That’s not something I’d like to scale with galaxy. So we have some super simple node scripts running on DO servers to handle those requests, and scaling those up is much more cost effective.
3 Likes
That sounds like a reasonable approach @vigorwebsolutions, offloading the heavy parts to a microservice hosted elsewhere. But it’s sounds like @orbyt simple node.js server for handling the chat socket traffic.
On that note @vigorwebsolutions, do you’ve suggestions/insights (perhaps point us somewhere) on how does your DO microservices servers communicate with the Meteor backend? are your mailgun servers completely independent or do they share info/ DB docs? if so how?
I’m still curious as to why several have mentioned that if I combine the current functionality in the bare node.js app with the meteor.js app (and thus be able to remove the node.js app entirely) that I may run into scaling issues.
Meteor.js is just a node.js app under the hood, is it not? Given the context of this thread, what differences would there be in scaling the barebones node.js app vs the meteor app?
I think it has to do with Meteor real-time pub/sub system which tails on the Mongo Oplog, and I’m not clear on when a fine tuned app that uses Meteor’s pub/sub start showing performance issues and how severe are those. I assume if you don’t use pub/sub then it’ll have the same scaling behavior as your typical node.js app.
You can follow the Redis-oplog thread for more info on how to solve scaling the pub/sub system.
I hope someone with firsthand experience on scaling Meteor pub/subs system can provide more insight, these are just my assumptions.
So we just dump the mailgun data into a collection that gets cleaned up and then aggregate that data to show an overall view on open rates, bounces, etc. The DO servers are very simple. We use MongoClient to connect to the DB, then listen for our webhook. run the signature to make sure it’s valid, then do an insert into the collection.
Got it, so you’re integrating at the DB level between the DO servers and the Meteor backend, thanks for the input @vigorwebsolutions.
1 Like