@voidale If pub/sub is dying, what are the alternatives?
MQTT is similar to pub/sub
1 Like
Looks interesting 
Thanks!
you have a source on pub/sub “dying”? MDG has said that they’re working on pub/sub in their Apollo work, which will have a nice implementation for Meteor
6 Likes
There seems to be a fad in the Meteor forums to declare that a Meteor feature or package is “dying” or on the way out simply because:
-
Someone doesn’t know how to use the feature properly or have the engineering/computer science background to design a system to be able to manage computational complexity or scalability.
-
The feature or package is stable and mature so it hasn’t been generating much news or required drastic updates.
-
Someone wants to promote an alternative technology that they have a vested interest in and are trying to manipulate people by creating self-fulfilling prophecies to kill off rivals.
Well guess what - my business has implemented a Meteor-based real-time vehicle tracking app utilising Blaze, Iron Router, DDP, Pub/Sub - features and packages that a vocal minority of misguided people seem to want to kill off or at least disseminate Fear, Uncertainty and Doubt (FUD) over.
I and other Meteor developers want this crap to stop.
I want the MDG and other package developers to know - you have done great work!
Your efforts are really appreciated and mean a lot to us. As the saying goes, hindsight is 20/20 but your design philosophy and implementation decisions have largely been correct. Meteor is a good product and is greater than the sum of its parts. Don’t listen to these whingers who are trying to make you second-guess, question all your decisions and abandon your previous work.
Our Meteor app runs 24hrs/day and has hundreds of drivers tracking in at least twice a minute whilst publishing real-time updates and reports to many connected clients. Yes, this means Pub/Sub and DDP.
This is easily being handled by a single Node.js process on our production server - an off-the-shelf Linux VPS running CentOS 7 - consuming a fraction of a single core’s available CPU power during peak periods and only several hundred megabytes of RAM.
How was this achieved?
We chose to use Meteor with MySQL instead of MongoDB. When using the Meteor MySQL package, reactivity is triggered by the MySQL binary log instead of the MongoDB oplog. The MySQL package provides finer-grained control over reactivity by allowing you to provide your own custom trigger functions.
Accordingly, we put a lot of thought into our MySQL schema design and coded our custom trigger functions to be selective as possible to prevent SQL queries from being needlessly executed and wasting CPU, IO and network bandwidth by publishing redundant updates to the client.
As we know, Meteor is tightly integrated with MongoDB which we did not consider to be well-suited to our needs. Did this result in us ranting publicly about how Meteor and/or MongoDB are “dying” or “having no future”? No!
Instead, we found a way to embrace and extend Meteor - an ongoing task.
In terms of scalability in general, are we limited to a single Node.js process? Absolutely not - we use Nginx to terminate the connection from the client and spread the load across multiple Node.js processes. Similarly, MySQL master-slave replication allows us to spread the load across a cluster of servers.
We look forward to additional performance gains through upcoming Node.js v6 support and other optimisations.
And React? Apollo? GraphQL? Even Flow router? Don’t have any need for them at the moment. Sorry.
47 Likes
Regarding the future of DDP, I’ll quote @debergalis from his Meteor 1.4 and beyond talk when he was talking about using DDP and Apollo side by side:
… we’re committed to a path where you’ll be able to use these different pieces together as we figure out the right time for a transition for Apollo to be the new default data layer.
I think MDG has made it pretty clear (well, clear if you attend Meteor Night or watch the videos) - for now nothing changes with DDP, but in the future expect it to be handed over to the community for support, as MDG themselves are going to be fully transitioning over to Apollo.
6 Likes
People who are asking if pub/sub is dying don’t understand the business value that pub/sub provides. Pub/sub isn’t going away for certain industries or certain types of applications. So, no, it’s not dying. And @vlsasky is spot on regarding inexperienced devs, stable packages, and FUD dissemination.
A better way to phrase the question is this: “Is the default example that MDG is going to ship in their base build going to be more of a public news aggregation site or a B2B CRUD app?”
13 Likes
@vlasky,
Enlightening post! Except maybe the Iron Router part
We are in the same exact shoes as you, small solid team that can build innovative products with the best tools at the time, and that makes improvements when packages don’t fulfill their needs. We haven’t embraced React (we have tried hard, we simply don’t like it and don’t see how it will make our life easier).
As Meteor packages become more community-driven, it will be the task of companies like ours and yours to take the lead on this. If Apollo doesn’t cut it, we will have to keep upgrading DDP, etc.
Question: We are excited to use your MySQL package. Are you using MongoDB and MySQL side by side? Your package has hardcoded ‘data’ in it, which means you don’t use if for everything (e.g. users).
2 Likes
Smart way of looking at it.
I watched this talk too. I agree that as far as MDG is concerned, they’re no longer going to build upon or support it.
Hopefully it, along with the interrelated technologies, will get turned over the right way. Then it will be up to the community if it “dies” or not.
There’s no way I’ll have the time move off of Meteor-classic anytime soon (but I will try), therefore have the responsibility of developing and maintaining a company built off this tech. It’s heartening to hear others in the same situation here on these forums – we have to stick together and support each other.
Will you share your experiences here? I mostly hear praise regarding React on this forum. At least from the people of note. I’d like to hear the other side from users that have actually tried it for themselves. I ask because I too will at some point try my hand at moving over, for reasons I and others have express elsewhere.
Before Meteor and Mongo, I used SQL exclusively for many years, I too would be interested in using MySQL and Mongo side-by-side.
I agree with this sentiment.
All these, “dying” posts, are really not very constructive – I for one learn very little from them.
Although, this post from you on this “dying” post has already, in a way, been beneficial to me. I wonder would you have posted anything like what you did otherwise. Sometimes they do have great posts like yours.
Also, they’re indeed abandoning their previous work in favor of a new tech strategy, but this does not take away from the great work they did (and continue to do).
I agree with this as well. I think they got it almost right the first time around. Although, I think internally they’ve already decided if they had to do it all over, they’d do it differently – or at least do it like Facebook – hence Apollo.
I too have a Meteor application being used 24hrs a day, but the application at this point doesn’t have the # of users. I’m glad to hear if and when I expand, there are options that will allow for it.
Have you open sourced your solution here?
How will you get there? Are you updating Meteor to 1.4 and beyond?
I too have been extremely happy with the Meteor-classic stack – I wonder sometimes if I don’t know what I’m missing with the Facebook stack.
1 Like
I think we are going through normal ‘quest for profitability’.
MDG’s first iteration (what @aadams has elegantly coined ‘meteor classic’) is a great product. A great product mostly for innovators and startups. It takes no time to have a great app out there. There is a design fault with oplog trailing (which we are looking into resolving with either RethinkDB or MySQL) which prevents scaling, but the rest is solid.
Larger companies however are (usually) NOT innovators. Microsoft purchases companies, IBM purchases companies (Google has some innovations but also purchases companies – seems their own Angular is not as widespread internally as you would expect). So companies like us who are pushing the boundaries and are focusing on MVP have other needs vs. larger established companies with legacy products, large teams, thick SOP’s, heavy legal etc. We have the luxury of being able to fork a repo and fix it, we can use newer web engines, spin up VM’s in under 5 mins. etc. Larger companies can’t always do that.
Where is the money for MDG? Larger companies of course. Sadly, I haven’t paid a cent yet for Meteor (not because I don’t want to, but Galaxy doesn’t work for our topology).
So we have to carry the torch of what MDG started with meteor classic where we find it fits our needs.
1 Like
If there are any prizes up for grabs here, I’m claiming “first coining” 
:
2 Likes
We didn’t jump on the React bandwagon as it added no value, but destroyed value for us.
React has the following benefits:
- Javacripts your view layer – if you are not a full stack developer or are just a JS developer, this is great news for you
- Forces a thinking paradigm where you compartmentalize
- Suitable for larger teams – not our case, and it slows things down
- Virtual DOM - at the expense of CPU and memory
- There is a trend in Meteor away from reactivity, and I personally found reactivity to be cumbersome the meteor-way with React (puns unintended).
- You can get a dev job easier if you are a freelancer – doesn’t apply to me personally (or to our team as learning React is easy)
- React-Native
In our case, we like Ember / Handlebars / Spacebars / Blaze (edit: and Vue, see next post ) approach where the view layer remains HTML (we like to think of it as XML ) We properly compartmentalized our templates, so everything is clean.
Blaze also does a good job of only rendering what needs to be rendered, so VirtualDOM may only add incremental benefits. And as it gets handled by the community (including us, we will invest in it) it will continue to progress.
So … why switch?
Edit: for example to show a list, we call this:
{> List data=listData header="main-header" description="main-description" onClick=listHandler}
And we use Tap-i18n for reactive translations – beautiful!
4 Likes
Post fixed - sorry about that
1 Like
Not offended - I was attempting humour
3 Likes
Interesting, please elaborate. We have performance problems with a graph heavy (dc.js) site with very few users. We only use a single Mongo but I’ve been considering setting up a replica set in order to enable oplog tailing. Would that actually be a good idea?