Smart way of looking at it.
I watched this talk too. I agree that as far as MDG is concerned, they’re no longer going to build upon or support it.
Hopefully it, along with the interrelated technologies, will get turned over the right way. Then it will be up to the community if it “dies” or not.
There’s no way I’ll have the time move off of Meteor-classic anytime soon (but I will try), therefore have the responsibility of developing and maintaining a company built off this tech. It’s heartening to hear others in the same situation here on these forums – we have to stick together and support each other. 
Will you share your experiences here? I mostly hear praise regarding React on this forum. At least from the people of note. I’d like to hear the other side from users that have actually tried it for themselves. I ask because I too will at some point try my hand at moving over, for reasons I and others have express elsewhere.
Before Meteor and Mongo, I used SQL exclusively for many years, I too would be interested in using MySQL and Mongo side-by-side.
I agree with this sentiment.
All these, “dying” posts, are really not very constructive – I for one learn very little from them.
Although, this post from you on this “dying” post has already, in a way, been beneficial to me. I wonder would you have posted anything like what you did otherwise. Sometimes they do have great posts like yours.
Also, they’re indeed abandoning their previous work in favor of a new tech strategy, but this does not take away from the great work they did (and continue to do).
I agree with this as well. I think they got it almost right the first time around. Although, I think internally they’ve already decided if they had to do it all over, they’d do it differently – or at least do it like Facebook – hence Apollo.
I too have a Meteor application being used 24hrs a day, but the application at this point doesn’t have the # of users. I’m glad to hear if and when I expand, there are options that will allow for it.
Have you open sourced your solution here?
How will you get there? Are you updating Meteor to 1.4 and beyond?
I too have been extremely happy with the Meteor-classic stack – I wonder sometimes if I don’t know what I’m missing with the Facebook stack.
1 Like
I think we are going through normal ‘quest for profitability’.
MDG’s first iteration (what @aadams has elegantly coined ‘meteor classic’) is a great product. A great product mostly for innovators and startups. It takes no time to have a great app out there. There is a design fault with oplog trailing (which we are looking into resolving with either RethinkDB or MySQL) which prevents scaling, but the rest is solid.
Larger companies however are (usually) NOT innovators. Microsoft purchases companies, IBM purchases companies (Google has some innovations but also purchases companies – seems their own Angular is not as widespread internally as you would expect). So companies like us who are pushing the boundaries and are focusing on MVP have other needs vs. larger established companies with legacy products, large teams, thick SOP’s, heavy legal etc. We have the luxury of being able to fork a repo and fix it, we can use newer web engines, spin up VM’s in under 5 mins. etc. Larger companies can’t always do that.
Where is the money for MDG? Larger companies of course. Sadly, I haven’t paid a cent yet for Meteor (not because I don’t want to, but Galaxy doesn’t work for our topology).
So we have to carry the torch of what MDG started with meteor classic where we find it fits our needs.
1 Like
If there are any prizes up for grabs here, I’m claiming “first coining” 
:
2 Likes
We didn’t jump on the React bandwagon as it added no value, but destroyed value for us.
React has the following benefits:
- Javacripts your view layer – if you are not a full stack developer or are just a JS developer, this is great news for you
- Forces a thinking paradigm where you compartmentalize
- Suitable for larger teams – not our case, and it slows things down
- Virtual DOM - at the expense of CPU and memory
- There is a trend in Meteor away from reactivity, and I personally found reactivity to be cumbersome the meteor-way with React (puns unintended).
- You can get a dev job easier if you are a freelancer – doesn’t apply to me personally (or to our team as learning React is easy)
- React-Native
In our case, we like Ember / Handlebars / Spacebars / Blaze (edit: and Vue, see next post ) approach where the view layer remains HTML (we like to think of it as XML ) We properly compartmentalized our templates, so everything is clean.
Blaze also does a good job of only rendering what needs to be rendered, so VirtualDOM may only add incremental benefits. And as it gets handled by the community (including us, we will invest in it) it will continue to progress.
So … why switch?
Edit: for example to show a list, we call this:
{> List data=listData header="main-header" description="main-description" onClick=listHandler}
And we use Tap-i18n for reactive translations – beautiful!
4 Likes
Post fixed - sorry about that
1 Like
Not offended - I was attempting humour
3 Likes
Interesting, please elaborate. We have performance problems with a graph heavy (dc.js) site with very few users. We only use a single Mongo but I’ve been considering setting up a replica set in order to enable oplog tailing. Would that actually be a good idea?
What performance problem are you referring to, it’s slow (i.e. polling) or your server is maxed out.
Note that you don’t need to create a new replica set per se, you can enable replica on the local instance of Mongo and get oplog.
It is very difficult to say where the slowness comes from. I wish I could say. I mean “slowness to render when you click on something”. Sometimes more than 2 seconds of seeing “loading …” when clicking something.
We had maxed out CPU issues for a long time until I moved to bigger machines on EC2.
In these situations the app became completely unresponsive. Now it is just “not snappy”
I currently have two meteor processes running on an EC2 t2.medium using mupx. They run on different ports and nginx “ip_hash” load balances on a sticky ip basis.
Mongo runs on a separate t2.small and never shows any load on the graphs on the EC2 console.
As mentioned, we do not have oplog tailing enabled.
We have separate node processes that write to / update the same collections so contention /locking issues here is not out of the question. On the other hand, simple admin pages reading from small collections also aren’t ever very snappy.
Is there a way of gaining insight into how much data is being transferred via ddp between the server and a client?
I’ve paid for the basic Kadira but this hasn’t given me any clues, nothing looks obviously wrong.
Server CPU profiles just showed a lot of time on “other” or something, and profiling of the app when it was flailing at 100%cpu never succeeded. Nothing helpful from client-side profiles either. From memory the bulk of time was within Meteor “processing Mongo documents”
Number of users seems more or less proportional to CPU usage. We have never even had number of users > 15. Users see a lot of graphs that can be resized etc so there are some unavoidably heavy subscriptions.
I’m using most of the suggested kadira packages except for kadirahq/subs-manager
This made things snappy but at the expense of up to date information.
If Pub/Sub/ddp got things totally right why would there be any need for subs-manager?
We have been running on Meteor 1.1 for a long time.
I have a 1.3.4 version of the app being tested in Staging. It doesn’t seem obviously faster.
I wrote the initial sketch of an app then it was fleshed out by a relatively junior developer.
There is definitely plenty of sub-optimal code in there, but I would like to be able to work out what parts to investigate from more of a “distance” first.
Any advice on diagnosing the slowness please?
METEOR@1.3.4
standard-app-packages
iron:router
accounts-password
multiply:iron-router-progress
ian:bootstrap-3
ian:accounts-ui-bootstrap-3
ephemer:reactive-datatables
wylio:mandrill
comerc:bs-datetimepicker
kadira:debug
meteorhacks:kadira-profiler
meteorhacks:fast-render
standard-minifier-css
standard-minifier-js
email
meteorhacks:ssr
.A
So first, a disclaimer, I am a full-stack developer, which means I have 80-85% of what an expert developer in their respective discipline has. So I’ll put on the had of a db admin on as my hunch is that this is where the problem is.
My experience is that db engines don’t always push the CPU through the roof when they are overloaded, they push ‘other’ processes and OS ones. MongoDB, even great as it is, requires frequent data to be in-memory (this is why they like SSD so much). They also recommend to remove from the OS last access time monitoring as it adds overhead.
So you need to start by diagnosing whether you have cache and index misses. There are a few tools out there to handle this. We use icinga2 as our monitoring tool, we set it and forget it. Now, you may need to probe deeper to see what is going on.
MongoDB often needs a lot of VM’s to run well by design (they claim it’s for redundancy). So it seems to be a built-in ‘feature’ that you need more machines and memory to get the job done
1 Like
Thanks Ramez, I’ll take a look at the Mongo side of things and let you known if I find anything or change any behaviour radically by changing things there. I may as well set up a replica set and two slaves. Would you expect that to change anything? I may as well try switching to oplog tailing?
Refer to this previous post of mine about using Meteor with MySQL
Yes, we successfully updated our app to Meteor 1.4, which required only minor changes to our code, and are now enjoying lower CPU usage mostly thanks to the Node.js v4 upgrade, which included a newer version of the V8 Javascript engine.
We can hardly wait to see what Node.js v6 will do for our Meteor app.
I would like to ask the 50 million dollar question:
Who chose to elevate Facebook to such supreme heights of holiness? Why are we expected to drink the poisoned kool-aid and accept the Facebook platform and libraries to represent the perfect vision of how our web apps should be designed or implemented?
Our app is not a Facebook clone or Social Networking tool and neither are most peoples’.
The Internet has been around for a long time and is used for many other things apart from social networking!
Sure I admire what Zuckerberg and Co have done, but that doesn’t mean I want to be exactly like them and do what they do!
To me, Meteor resembles Coca Cola. The company succumbed to unfounded FUD about their original formula and replaced it with New Coke, only for customers to realise that new was not necessarily better and that Coca Cola Classic was right formula all along - just like Meteor Classic.
MDG - are you listening?
2 Likes
I would not characterise oplog or binary trailing as a design fault - it is a valid technique for implementing reactivity, but it will reach a performance limit.
The rate of reactivity is limited by the speed at which Meteor can read the oplog/binary log, execute reactive code and push relevant updates to the client.
As long as this limit is greater than what is needed for your app, then there is no problem.
As you know, I’m using MySQL instead of MongoDB with Meteor as I find it performs better for our app. I have been continually finding ways to optimise our design and implementation to raise the performance limit higher and higher for a single server to minimise the size of our cluster.
We only looked at Meteor because it had pub/sub, i.e. realtime sync that allowed for very complex business logic. (Yep, it’s B2B.)
We liked Node because that reduced the number of languages, and Meteor checked that box.
We figured Mongo, or something similar, because we need a flexible record/doc structure for a few tables/collections — and Meteor checked that box.
We needed a build process, and Meteor checked that box as well. (And 1.3 has gone a long way toward cleaning up Meteor’s failings there.)
We needed users and some form of self-managed accounts, and Meteor helped out there too. (The profile subdoc is annoying, but livable.)
I could do without Blaze, or React for that matter (which I consider a step backward as it seems the JS alternative to feeding PHP to the browser). We were porting a large Knockout codeset and, for better or for worse, I find the Handlebars syntax bloated and I’m not a big fan of how it manages the DOM thru strings, so we only use it to bootstrap the page(s). At some point, we’ll factor it out.
Meteor has its warts and its weaknesses, but pub/sub and DDP certainly aren’t among them. I get that MDG is integrating Apollo in order to expand the architectures it can support, but I like how light DDP is and how I can write many of the business rules once, attach them to a transform (model) on the collection and use them on both server and client. I look forward to a long and healthy life for pub/sub and DDP
8 Likes
@vlasky,
Thanks for your comment.
The reason I believe it’s a design fault is as follows:
Imagine your app scales up quite a bit. With that scaling, oplog is going to scale up as well.
So each new user will result in increase in oplog changes. This will slow down ALL your instances of meteor. So you are no longer scale up by adding new instances for new users only, you also scale up by adding more instances as your current instances are getting overloaded! I can’t find an excuse for such a lousy implementation to be honest. It’s small-time reactivity.
In fact, mathematically, there could be a point where you can’t scale up any more, and any new user will result in automatic degradation regardless of number of instances.
Now, concerning your mysql solution, it is better than Mongo as we had discussed, as are postponing the breaking point:
- You set the update interval manually – which you can increase to reduce CPU hits, but at some point the app is no longer reactive real-time
- You set more complex rules to filter the binlog – then again, those rules have to be executed for EACH subscription, for EACH binlog change, which will eventually hit the limit too.
Solution: The DB itself has to be a realtime DB that is designed for reactivity. The only popular / robust one we can find is rethinkDB. We are working on the Mongo replacement driver as we speak. But all these client-based solutions will suffer at some point regardless of how you implement it, as you are (1) using a scripting language – JS, (2) parsing a lot of data to find what is relevant for you.
2 Likes