Hello,
I am currently creating a game with meteor + react and I need to sort out some questions regarding the data distribution, especially in terms of scaling.
Imagine a game like WoW with 100 users in the same place.
There are multiple collections per user, but only the userProfile (name, pic, …) & userTranslation (current position, current rotation) have to be reactive.
- userProfile doesn´t change that often, but each profile is big
- userTranslation changes very! often, but is very small
So, currently, if 1 user moves, the other 99 will each receive an update from the subscription to the userTranslations. Each user will receive the 99 other profiles and will then need to filter for the one that actually changed.
→ How can I achieve that each client only receives the data (reactive) from the one user that changed (without client-side comparison)?
My current subscription:
const userTranslationsLoading = useTracker(() => {
const subscription = Meteor.subscribe('userTranslations');
return !subscription.ready();
}, []);
const userTranslations = useTracker(() => UserTranslations.find({_id: {$ne: userId}}).fetch(), [userId]);
useEffect(() => {
if (!userTranslationsLoading && userTranslations?.length) {
const arrToObj = {};
userTranslations.forEach(usr => arrToObj[usr._id] = {...usr});
setUserTranslations(arrToObj); // save to global State (Zustand)
}
}, [setUserTranslations, userTranslations, userTranslationsLoading]);
return null;
Options I am considering:
1. adding a lastChanged to the publication / subscription
const userTranslations = useTracker(() => UserTranslations.find({
$and: [
{lastChanged: {$gt: localLastChanged}},
{_id: {$ne: userId}},
],
}).fetch(), [userId, localLastChanged]);
Pro:
- The clients don´t need to do any comparisons anymore - it will only receive data from the server if it is more recent than its local data.
Contra:
- For each movement, 99 new publications and subscriptions will be created, which is probably quite inefficient and heavy for the server? Movement is on mouse click and people like to click…
However these 99 new subs would all be the same and therefore reused on the server?
2. lastChanged + useTracker + Meteor.Methods
useTracker will inform each client that something changed, but won´t send actual data. It does however execute a method (with the localLastChange) which returns the desired data.
I already do it for the userProfiles - its working quite well, but yet again, I am not sure of its efficiency in terms of scaling or even if its hacky or not?
// ..Client .. subscription as in first example
useEffect(() => {
if (!userTranslationsLoading && userTranslations?.length) {
Meteor.call('userTranslations.getNewData', localLastChange, (error, res) => {
if (res) setUserTranslations(res); // save to global State (Zustand)
});
}
}, [setUserTranslations, userTranslations, userTranslationsLoading]);
return null;
// server
'userTranslations.getNewData'(localLastChange) {
return userTranslations.find({$gt: localLastChanged}).fetch() // pseudocode
}
pro:
- Client is happy without comparisons. Server is happy because its not bombarded with new pubs/subs?
con:
- probably also inefficient for the server since methods are not being cached serverside. I could use meteor-fast-methods but then I would need to go for redis?
- Probably a “triple strain” for the server - for each movement, 99 users will receive an update in the tracker, these 99 will then send a request to the server which then has to answer 99 times.
Browsing through the forum, similar solutions were mentioned a few times, most recently @minhna in this thread
Unfortunately I did not really find any best practices / examples regarding this.
3. Replace pub/sub and useTracker with websockets?
The app already uses custom websockets for other purposes. It would probably be a great way to distribute the positional updates to each client. However I still need the data in the database. It would make a few things much more easy (and others more difficult…) but it would still be a solution on top of the database.
4. Make use of redis-oplog or rather vent?
vent allows you to send custom events from the server to the clients that subscribe to it. These messages will not be stored in any way on the server. You may need this if you want absolute control over the reactivity.
Sounds great actually, but I am not sure if it will solve this problem. However from what I gathered so far from this forum, it really seems to be a magical solution and I will probably want it anyways…
Sidenote: I want to switch from Zustand to React-query for all the server related state. This will probably lead to a query based approach, making more use of meteor methods. But I still need that reactivity somehow.
Anyways, I am super curious to hear what strategy you would pick or maybe you find an entirely different approach!