I have read a series of posts on this forum now regarding realtime messaging between clients and wanted to get some advice.
What is the current best solution for ‘Real time’ messaging within the Meteor system?
My understanding is we have:
1. Collections
I get it collections are great, easy to setup and use and on the face of of perfect for a messaging API. Is a system based upon a collection per room going to run into performance issues quickly?
2. Websockets
My understanding is that Meteor uses Websockets anyway to implement realtime updates to minimongo, is there an advantage to seperating the concerns and implementing a seperate connection to say a node bases redis pub/sub?
3. WebRTC
This has popped up a couple of times in the forums as a option and favoured due to the direct connection between clients. But in this case how is chat history stored and connection dropouts dealt with?
The more I investigate, the more it seems that although simple on the surface, implementing realtime chat outside collections is quite complex. Correct me if I am wrong but I need:
A client side cache with pub/sub system capable of dealing with dropouts.
A server dealing with multiple connections and publishing filtered messages.
A database with the ability to store but quickly retrieve a potentially huge collection of small strings.
A media storage system linked to the server for media uploads.
This potential headache brings me back to Collections as a great solution, all I would need to add is a S3 bucket.
I am however worried about performance and do not want to get bitten by slow downs down the line.
Please could someone who has figured this out give me some pointers, thank you.
Technically you can use Redis and its channels to push real time messages outside of mongo. Whenever you get an update in a channel you could populate a client side collection via the publish api.
So I tried to draw this in paint and gave up because I cant draw. I’m just going to spell this out, let me know if it makes sense:
Meteor Server
A publish method (lets say ‘chats’) with an observeChanges hook on a Redis channel.
I could use the userId to filter the channels.
Meteor Client
Subscribe to ‘chats’ within a Tracker function.
Send messages within a method with auth/schema check, the initial message in a chat creating a new Redis channel. I could track participants in a chat with mongo on the server, maybe create a field on the user object which tracks collections user is part of?
My question really is this seems like a huge pain in the behind when I could just use collections. Am I going to get much better performance with this? I am heavily into Redux so am wondering if I would be better off just using something like https://pusher.com/ and connecting it to the store.
It really depends on how many users are online on the same time. MongoDB works fine but you can end up in oplog scaling issues if you have heavy writes due to added chat messages or status changes. If you have a room chat system with a small number of rooms, you’ll benefit from a high observer reuse, which is good for your server performance.
So for example, I would add all chat messages to a MongoDB collection, but temporary messages like “User A is writing…” to RedisDB - or you use a seperate Socket.io server for it.
For the future, Apollo could solve your problem. In this case RethinkDB (which was designed for realtime messages) may be a good choice.
@XTA thanks for reply, this is exactly the info I was after.
I have been experimenting with apollo, the trick there is implementing reactivity. I suppose regular polling would be sufficient and deal with disconnection reasonably easily.
For the user typing scenario would you just have a user channel in redis with states? I see that would take the hard work away from a collection, I know that we are limited to a few sockets (less with IE) but the separation of typing state and messages makes sence and can be accomplished with separate socket for state and apollo connection for message delivery.
I have to say that we currently also saving the state information to MongoDB (we update the user document if the user is typing or stops typing). At the moment, everything works fine with about 40k messages per day, running on a single dedicated server - but our Kadira logs show us, that we may should change this behavior if messages and users increase (at the moment we have a method response time of 100ms).
In your case I would use a channel per room and add all information to it (like user id 123456 is typing / stopped typing). Then you also subscribe via Meteor to all users within the room and can transform the user id of the Redis document to the user name. I didn’t test it yet, but you can also use Redis via Meteor https://github.com/meteor/redis-livedata
Great thanks 40k a day is a pretty good benchmark for my use. The addition of Redis makes sence for future and should be fairly easy to implement down the line.