Full-stack reactivity: opt-in or opt-out?

community
42

And this is exactly where we differ, I see Uber and the like to be future of many apps especially since many sensors/robots will be plugged to the web (IoT), that is why I voted for reactive by default and opt-out option and not the other way around… the fact that we don’t have many apps like Uber because it just recently that this technology is becoming easier…just 10 years ago we only had static pages on the web…

Pokemon Go is also heavily reactive…pretty much any app that users are interacting in real-time with others and the environment will need to be reactive, this is a space yet to be explored fully by entrepreneurs…that is why I voted reactive by default and opt-out option, since I believe it’s a more forward thinking strategy.

3 Likes
43

Personally, I believe a non-reactive app is nothing more than a static website, not an App. The reason we love Meteor, is for its live data on the wire. I like the examples in this thread, PokemonGo, Uber etc. We are here to build great dynamic user experiences, not websites.

8 Likes
44

I get your point. It’s independent from the point I was making. One of the strategic goals for Apollo should be to win developer mindshare. If developers are more comfortable with non-reactive apps, and Relay (the competition) is non-reactive, why not make this the default? Then they can say “hey, by the way we also offer reactivity” but not shove down your throat. Don’t you think this could be more strategic to win developers over?

Ultimately I vote for simplicity. I believe that libraries and frameworks that are simple, easy to understand, and efficient are more successful. React clearly won over Angular 2. Jquery was also simpler, less magical than other competing libraries: http://blog.teamtreehouse.com/jquery-popular-javascript-library.

45

Sorry, I find this discussion incredibly silly.

In our Meteor app, all server data that needs to be reactive is published. The client simply subscribes to those publications it needs to get updates whenever they occur.

All other data that is non-reactive can be retrieved at the client’s leisure by calling Meteor methods.

Also, I strongly reject the suggestion that the correct use of reactive data sources results in any kind of application performance penalty.

For our vehicle tracking app, we use publications to avoid clients needing to needlessly poll for updates, especially when a vehicle is parked or stationary for long periods of time. This saves much bandwidth and Node.JS CPU time.

4 Likes
46

The Meteor MySQL driver has a configuration setting “minInterval” which provides such a rate-limiting mechanism.

Such a mechanism could theoretically be implemented in the MongoDB driver too.

1 Like
47

No I don’t think this particular discussion is silly at all, I personally rarely spend time on the forms but reactivity along with the amazing developer experience are why I picked up Metoer in the first place…if this poll could impact the future design of the product and one of it’s main selling features (at least for me), then I think it should be discussed thoroughly…

What was your answer on this poll? You said you build a vehicle tracking app so I assume you made a good use of Meteor reactivity…the question here is not whether you can implement reactivity, I mean you can do that with plain web sockets and Java…the essence of the question is how should the framework present itself to first time users…what was your answer to that?

48

That is a very good point venwao, and I see where you coming from specifically pertaining to positioning Apollo…please correct me if I’m wrong, but my understanding is that GraphQL and Relay are still working on real-time syncing and it’s has not been finalized in the protocol…so there is no way with the current state of that technology to default and advertise reactivity out of the box (is that true?)…it might be wise to pace with the audience reception for Apollo and the community as a whole but from a ‘Meteor classic’ developer perspective (and I don’t claim to represent all) it might seem like a step backward…

I’ll rest my point here, I think I made my case clear, and it’s not my intention to waste others time or to be argumentative but I’m a big fan of Meteor’s reactivity, I think it’s the future, and it should be defaulted on ! :slight_smile:

3 Likes
49

I choose reactivity
Why then need Meteor without its magic?!
I chose Meteor because of its reactivity, ease of implementation and fantastic magic…

3 Likes
50

Guys for the record this poll isn’t about changing anything in meteor. Meteor isn’t changing.

I’m talking about frameworks in general, and trying to get feedback from people who might have tried both approaches, what see which worked better from different point of view.

1 Like
51

I think it’s a little confusing when you say “Meteor isn’t changing”, and I’m trying to understand in my mind what you mean. I’m thinking of Apollo as Meteor 2.0. I think that’s the general perception. You compared Apollo to Livedata 2.0 in another thread. But it’s hard to pin down what is really “Meteor”.

However I basically trust MDG, that you will create an easy solution for developers to do reactivity and non-reactivity, and create a simple blazed path (pun intended) for transitioning when it’s necessary to transition to new technologies. There is always some pain when technologies evolve and you have transition, but it’s great to have an organization like MDG who’s thinking about the big picture, not just creating libraries but also patterns and best practices.

1 Like
52

@sashko, from reading your comments, it seems like you’re saying

``> If we go reactive opt-out, we’ll be more likely to end up designing our interfaces around things being reactive – and we will have a hard time taking reactivity out. And if or when the system becomes un-performant due to server load brought on by reactivity, we’ll be in a hard spot.

whereas,

``> If we go reactive opt-in, we’ll be more likely to end up designing our interfaces around things being non-reactive. But we much less likely to encounter performance issues because because the server won’t have to hold on to costly sockets.

And the part I’m wondering about, do you think it will be easer to add reactivity in as needed in this scenario or do you think it will just as hard to add reactivity in that take reactivity out?

From MDG’s perspective, you want reactivity off by default because there will be less scalability issues to listen about, period. If you ship with reactivity on by default, someone can always point the finger back at MDG.

53

I think the same reasoning applies from the developer’s perspective as well.

54

I think it’s a reasonable perspective for MDG to hold.

55

Does it matter whether it’s opt-in or opt-out? Can we have it both ways? Unless I’m missing something, the idea would be that it’s just a flag set on a apollo endpoint, right? Similar to [Meteor.Collection].find({}, { reactive: false });.

If we just had a meteor.config file that said (or maybe just an environment variable):

reactiveDefault: false

We get the best of both worlds, no? The people that want to opt out only have to remember when to opt out and vice versa for opting in.

3 Likes
56

I think it’s not that simple, especially since not all backends support something like live queries. When do you re-run the queries on the backend? Of course, you could re-run all of the queries all of the time. But that’s not scalable at all.

So what do you do if your backend API is not reactive, or if you want to control when stuff updates?

57

With MongoDB, we monitor oplog, with MySQL (@vlasky’s library) we monitor binlog, with rethinkDB, it’s an actual cursor. Not sure about the other DB’s, but personally, I think we should be using a real reactive DB (such as rethinkDB) when we want real-time reactivity. The log monitoring approach is not scalable – has to be at DB-level

That being said, RethinkDB has two bad things right now:

  1. A complicated query language – we tried writing a Mongo to ReQL converter (for most common queries), but there are so many edge cases – almost done though
  2. There is no built-in index optimizer. In other words, you have to specify in your query which index you are using (and line up its fields)
1 Like
58

@ramez I also noted the lack of an optimizer in RethinkDB, as well as no support for unique secondary indexes.

Regarding reactivity triggered by oplogs & binary logs, I don’t think they are as bad or limiting as you think.

Having thought a lot about this, I have come up with a rough guideline to determine if there will be a scalability issue:

  • If you can successfully set up a MongoDB or MySQL slave database that can keep up with the master database server used for your application, you can theoretically create a reactive Meteor app that can handle that level of load on the master database.

Explanation:

Oplogs & binary logs were intended to be used for implementing asynchronous master-slave replication. The slave database server reads the oplog/binary log produced by the master in real-time and uses it to update its copy of the data.

In order for a master-slave replicated setup to be fit for its purpose, the slave server has to be able to keep up with rate of updates from the master, otherwise the slave lag will continually increase and the slave can no longer be relied upon to be up to date with the master.

We can presume that everyone who is using master-slave replication has a working setup. That is, their slave (on average) keeps up with the master.

My experience and dealings with MongoDB enterprise users are minimal, but in the case of MySQL, master-slave replication is very popular and is used for very busy production servers. Most often, there is only a single thread on the slave MySQL instance responsible for reading the binary log - just like with NodeJS, which is effectively a single-threaded environment.

Hence, the resource demands of reading the binary log are normally not the performance bottleneck in a replicated setup. Bottlenecks are almost always related to the speed/number of CPUs on the server or max I/O rates of the storage subsystem.

So the important question is: How does the demand on system resources differ between:

  • A slave database that reads the MongoDB oplog or MySQL binary log

  • A Meteor app with reactive publications triggered by the MongoDB oplog or MySQL binary log events.

A Meteor app with reactive publications is similar to a slave database server, except instead of updating the slave database, it triggers the publication of data and sends it to connected clients.

Unlike a slave database server, the Meteor app does not have the overhead of writing to the disk. However, the Meteor app has the overhead of frequently having to make database queries, package the data inside the DDP layer and then send it to all connected clients that subscribed to the published reactive data source.

For this reason, all of the scalability improvements we have implemented in our Meteor MySQL-based app have been achieved by finding ways to speed up the query execution, minimising the amount of data that has to be transmitted to the client and spreading the Meteor’s CPU load over multiple NodeJS processes.

Also, our servers use pure SSD storage, which makes the binary log file access much faster.

6 Likes
59

The map might be 100% of the view, but it’s 1% of the app. The other 99% is in their matchmaking algorithms. And this, too, requires real-time data. Showing someone an updating car moving around a map is not nearly as difficult as getting the person the car in the first place. Uber needs to send push data to their users to notify them of new drivers!

60

I think it depends what kind app you are building.

for Admin/Dashboard/Chat/Helpdesk Desktop WebApp: default reactivity ON (i will use Meteor)

for React Native/Cordova Mobile WebApp: default reactivity OFF & Offline First (i will use Apollo + Redux/PouchDB)

My understanding is Apollo should be a helper for Meteor to provide offline first API to mobile client, rather than a total replacement to it.

61

Trying very hard not to be an MDG apologist here:

Again, its seems reasonable for MDG to hold this position. But one of my problems is not going reactivity first, it’s like MDG is giving up somehow.

Does MDG think GraphQL, Apollo, will NOT be scalable going fully reactive under load?

1 Like