One of the shortcomings is that scaling has never really been addressed by MDG. The community contributed a lot of stuff like cluster but that was to be honest a hack. Even Galaxy doesn’t have autoscaling. Setting up proper scaling with a Meteor app is not trivial, to be fair its not trivial for regular apps as well, but this is something that should be addressed.
4 Likes
@evolross, for social apps I’m assuming you typically need realtime features for things like presence? How reactive does your app need to be?
There are a few libraries/frameworks, but none of them are anywhere close to Meteor in terms of completeness/polish -
Deepstream.io
SocketCluster
FeathersJs
FeathersJS and Loopback. I prefer Feathers due to its easy setup. If you’re looking to include React, Redux and Feathers use this GitHub - bertho-zero/react-redux-universal-hot-example: A starter boilerplate for a universal webapp using react, redux, express and feathers.
Michel
Presently, reactivity isn’t really needed for the hundreds/thousands of event users, there’s design approaches to work around that. Reactivity is needed in collating all the incoming data from the massive amount of users, but that’s only on a few admin-use-case interfaces used by only a few users (not hundreds or thousands).
I need to still experiment with setting the pubs/subs of the mass users to non-reactive and see if that reduces the server load, presently they are not, even though they’re pulling very little data. The new code splitting will help with this too I hope. Presently I’m running separate apps for each use-case (massive users and admin users). But there’s some extra unnecessary overhead with this and the code each app shares.
Anyway, was just curious if people had any suggestions. FeatherJS looks pretty young, as you say.
1 Like
This is what I’d advise, these are some basic principles you can adapt as needed.
- make your app server stateless, this allows you to scale out easily as needed, and to kill/restart servers
- to do this you can keep session state in a cooki or token (JWT), or you can use a session store like redis. e.g. https://github.com/smrchy/redis-sessions
There are also redis adapters for the frameworks I mentioned - to get the data, start with a simple REST api, and use redis as middleware to cache if needed. The same cache can be used my multiple servers
- use Redux on client as state store and cache if you plan to use React. Then you can also use SSR and hydrate store for fast initial loads
- run microservices, again these can be scaled out if needed.
- for the parts that need to be reactive, use channels (event in deepstream), named endpoints which you can publish data to. You can also make a particular object realtime by subscribing to it.
The main idea is to to make it easy to spawn servers and don’t make each server do too much, so its easy to find out perf issues. Can also use http2, cdn for static assets etc, and use monitoring/logging everywhere.
This kind of composability is impossible with a monolithic framework like Meteor. Doing it by choosing each layer is going to be harder but you can control a lot more.
6 Likes
After using meteor for 2 years, we have now built 2 projects with https://github.com/react-boilerplate/react-boilerplate and are building our main product around this repository.
I’m learning Elm 
It has lots of the automation in-built as Meteor does, so I don’t need to go crazy to setup the dev environment.
It makes my Javascript MUCH better than it could ever be on its own (me not being a super knowledgeable JS guru and all).
And I don’t need to bother with frameworks on the front end side of things.
I’m still at the beginning, but I’m liking a lot what I see 
2 Likes
bud - same place. mobx for the win.
2 Likes
Can i ask - i was looking at a model where large server side functions are moved to the google/aws/ms compute engines? Any initial thoughts? comment appreciated…
1 Like
This makes a ton of sense in some situations. I’m using AWS Lambda, for example, to process uncompressed audio files that people upload and run ffmpeg on them to transcode to AAC. Lambda is also good for large-scale image resizing too.
rethinkdb
fusebox
react
mobx
nextjs ?
nice… i’m using azure for a lot of other stuff, but will give it a shot with azure functions…
presumably all long running tasks would be best serviced out of this? like, batch send emails, or batch create pdf’s, or whatever?
given its just a function, how do you include other npm libs here, and does it work as you would expect? noob example would be appreciated? Also, can you get this to talk to the db?
thanks mate…
edit: does it make a difference to the meteor deployment? say i’m on galaxy, and start handling all out this out of the compute engine. where exactly is the difference. like, would this cost me less?
i’m missing context info, so getting confused. if you can explain your use case without giving away anything proprietary, would be most appreciated.
Thanks for this, didn’t know about this and it looks great.
React/Apollo/Elixir I think…
An AWS Lambda function is nothing more than a bit of code that gets executed. It can be Python, Node.js, whatever. So with Node, it’s like any other script you’d write. You can require packages (you bundle up node_modules in the zip file when you create your Lambda function) and you can do whatever you want, pretty much.
1 Like
Redux creates a lot of indirection, and itself can have odd side effects. Like I have a Componet that is a grid, and you can select an item in the grid. This selected index is held in a Redux store. Now, when the user clicks to edit a different grid object, the system uses the same grid. Because Redux is decoupled, the selected index is unchanged, and when the new grid is rendered, it renders with the old grid item selection.
This isn’t terrible, but if I had used local state, then all I would have to do is reset the key on the grid editor instance, and the state would be reset. If I use Redux, I have to use lifecycle methods to reset the essentially global state variables (Redux is basically an interface to a global object store).
I used to use Redux to store all my data. But if I use Meteor data connectors (or Apollo), then they have already implemented various subscription optimizations to unsubscribe from data sources, and clean up member a bit (presumably). If I put everything in my Redux store I have to manage all that manually. This has pros and cons.
I may yet go back to Redux and use some of those fancy offline libs, but I’m really on the fence about it. The redirection in Redux does cause some headaches. I can still get unidirectional data flow with Meteor’s connectors, and it actually feels cleaner that way, easier even to reason about. Maybe it’s just me though.
3 Likes
i’m sorry - i probably don’t understand how to use redux, or the use case, or something, but for my money it is a piece of shit.
Maybe massive benefits when it comes to scaling or something, but i don’t have that problem to the extent non-stupid design could not fix it…
Wow. 
You might find no personal use for Redux, but if it were truly a “piece of shit,” I doubt it would have as many people and companies using it. Granted, I have my own opinions about Redux (I think it’s maybe a little overly complex), but I can still see its value. Now, as far as using it in the Meteor ecosystem… you’re right, it probably has very little use since we can use ReactiveVar and Session to handle global state. In fact I’ve ripped out Redux from two Meteor projects that I thought I needed it in. But for those who are using pure React as their front-end, Redux or Mobx is—in many cases—totally necessary.
It seems to me that if you’ve a really large and complex client with a lot of things changing, many dependencies and multiple people collaborating on the code base, then I think it’s useful to be very explicit, transparent and develop a centralized and global source of truth on how each action changes the entire state of the client app.
For medium and smaller size apps, where one or two developers working on the front-end code and state changes are not that complex to grasp and maintain in the developers memory, then something like Redux might be an overkill. Furthermore Meteor does have mini-mongo and reactive data sources but it leaves it up to the developer to decide where and how this data will change. If the app grows in size and the developer continues modifying global reactive session variables all over the code, then you might end up with unexpected states, bugs that are hard to trace, and unmaintainable code.
But yeah I don’t like it when people blindly follow the latest trend and adopt the next shiny toy out there without careful evaluation of what does it really provide and whether it’s the right tool for the job in hand.
2 Likes