Out of curiosity, is it more faster and/or more scalable than meteor pub/sub with redis-oplog?
No worries! I totally agree with you. There needs to be this kind of conversation. That said I’m not abandoning Meteor as a platform entirely, just migrating this particular app off. I fully intend to continue to dev in Meteor for other projects.
3 Likes
I’d like to chime in with my experience with meteor too. What kept us this long on meteor:
- redis-oplog
- the fantastic build tool meteor has
- hot reload
- Ddp (no reason for us to jump on graphql as mongo is working well for us, and besides, reactivity isn’t top of mind for graphql)
We built a chrome extension with Webpack using a ddp client and it works great. Once we figure out how to have our own reliable ddp server and integrate with redis pub/sub we’re gone too.
Meteor isn’t that bad - it’s just not how we build apps anymore. If you are an experienced dev who wants to customize your app, you can build a much leaner and faster product yourself. Especially with React (we use mobx and love it).
Once we dropped Blaze - oh man! We felt alive again. No more defers and jquery
4 Likes
@veered, maybe that’s what we need, a visible corporate sponsor.
I recall two efforts I attempted to help Meteor, one was a community fork a couple of years ago on which MDG jumped on and put the brakes, and the second was a discussion with some active members of the community to see if we can do something. Needless to say, both failed. So a visible corporate sponsor (we can jump in soon) may be the way out of MDG’s complacence. It’s obvious their goal is to milk Galaxy to finance Apollo – we also need to ask if it is not wise to use Go for the server.
On another note, we did something I believe is really cool, move mergebox client-side. We used @mitar 's disable mergebox package and we do the merge client-side. Redis-oplog doubled our user count per server, I believe we can achieve another 2x with this approach
Finally, we use ElasticBeanstalk for hosting – doesn’t cost much and is scalable. We should advertise more as an alternative to the community. I was holding back so not to hurt Galaxy, but at this point, who cares?
5 Likes
This is becoming a competition among NodeJS build tools, when people migrate they’re simply swapping the build tool. Currently, there are three major build tools within the node ecosystem (webpack, parcel and rollup). Meteor started as a full-stack but it’s evolving to be one of the best build tools, in my opinion, thanks largely to Ben’s effort and also the community feedback/usage/packages over the years.
Why do I prefer it over other build tools? Largely because of the philosophy of backward compatibility, many features of the box, community and the zero-configuration yet incredible flexibility. You can see just from this thread the various ways people are using Meteor and the variety of use cases its addressing. This balance between opinionation and flexibility is really hard to achieve and out of all the existing build tool Meteor is the most balanced. I also really believe that NodeJS ecosystem needs a tool of that sort to enable entrepreneurs.
However there are things to improve. There are patterns of what people seem to look for when selecting a build tool:
-
The developer experience (HMR, startup times etc.)
-
The bundle size optimization (tree shaking, dynamic imports, modern bundles, visualization & analysis etc.)
-
Longevity (since the build tool is becoming foundational tech)
-
Features (what comes out of the box, the packaging system, etc.)
I really hope MDG and the community keep doubling down on making the build tool in Meteor more competitive by closing the gap with competitors (tree shaking, HMR/faster refresh and better sponsorship/marketing).
For branding/longevity, one strategy would be to extract the meteor build tools, add Apollo to it out of the box (that can be removed), keep it backward compatible with existing meteor packages (blaze, ddp etc.) and rebrand it as Apollo CLI/Build tool. More and more people are using Meteor that way anyway so it’s reasonable to make it the default, also by branding it as an Apollo build tool, Meteor will get the benefit of the marketing effort going to Apollo.
I’m actually surprised this has not happened yet, I think Meteor build tools should be the easiest way to get an Apollo powered app going (mind you I still don’t use Apollo and I really enjoy the simplicity of Meteor methods). This strategy will surely give people the assurances they seem to seek and it’ll also keep the folks using Blaze/DDP happy because the core foundation will keep on improving. From MDG perspective, they’ll only have to market the Apollo platform (and include the re-branded/extracted Meteor tools as part of their DevTools offering) which will indirectly results in more Galaxy deployments/revenue something MDG invested a lot on, it seems like a win/win to me, although I do think bottom up marketing is also good (starting with the build tool) since this is where a lot of developers start. Anyway these are just my opinion/thoughts … also hire captainn heh.
7 Likes
Hi @stolinski
As for your original question - we, The Guild, have been helping many companies migrate Meteor applications.
Some gradually, some as a rewrite and would love to help you with the migration.
It’s even just worth chatting before you start so we could share with you different possible paths and maybe recommend the right path according to your needs, with whoever contractor you might choose.
I won’t go into the other parts of the discussion, maybe some day we’ll publish a blog post about our position and experiences.
We still maintain some Meteor packages (some of them help with gradual migration like meteor-webpack and meteor-client-bundler) and Meteor production applications, but we’ve moved on to the general Node and GraphQL ecosystem long time ago, maintaining a lot of Meteor agnostic open source packages
@alawi, as always, your posts are well-written and well thought out.
My only 2 cents are:
-
Some of the build tools you mention are really good. Webpack for example, while being clunky, really allows deep customization of your build. It’s hard to see space for another build tool. It’s one of those things that Meteor did very well and did it first, but was not properly advertised or showcased. I think it has to do with the single-message problem. You need to focus on one message and MDG focused on isomorphic simple-to-develop platform and that stuck
-
I believe MDG makes its revenues from the Apollo Engine, Galaxy is a legacy product that they are milking as long as it lasts. But it’s not their future
In other words, even if it’s a great product, there is now little commercial incentive or share of mind for developers left; and now that we are past a critical point, there is little open-source incentive with other amazing solutions out there (next.js, gatsbyjs etc. – of course, React-based).
3 Likes
Thanks @ramez for reading through.
For the first point, yes I agree the bundlers space is competitive and I think Parcel is closer competitor to Meteor than webpack. However I think Meteor can be positioned as an Apollo backend with a rich ecosystem (accounts, etc) and flexible front-end as opposed to just a bundler. Something between the minimalism of Parcel, the opinionation of CRA, Vue CLI and NextJS and the complexity and freedom of webpack.
For the 2nd point, yes I can see Galaxy is being milked for revenue but if Apollo is used within Meteor projects by default, then Galaxy will be be a graphql platform as a service that comes with their Apollo Engine by default. I see Meteor/Apollo to be complementary products and I think they should be marketed as such and this is not very surprising given that Apollo was created from the lessons derived out of creating Meteor’s DDP data layer.
Anyway, folks at MDG are smart and I’m sure they’re aware of this possibility and they have their reasons for not doing it thus far.
2 Likes
From hardware perspective, no difference, or minor. From development perspective it’s more scale-able because of GraphQL.
1 Like
Totally agree.
IMO Meteor’s competition isn’t Webpack and Parcel itself, but the things built on top of it aka Next.js and Gatsby and others.
If we could find someone really good that’s active in the Meteor open-source community, we would probably hire them to primarily work on open-sourcing some of our stuff and building out important things for the community in general.
I reached out to @mitar but he’s had his hands full with his PhD! I’d love to find someone though.
Edit: We have a strong preference for on-site candidates in San Francisco.
10 Likes
Y’all should take a look at Waves Hosting to see if it’d be a good fit.
I was gonna say, why compare it to webpack when you have tools like NextJS or Gatsby where you ever need to change webpack config?
I’m that sense, I’m curious, @stolinski why do you need someone to know webpack config? If you are not using something like NextJS or Gatsby, why is the reason behind that? If you are, what kind of config do you need to extra config that doesn’t come as default?
I wish that if the future of Meteor is really a better bundle tool + great added features, MDG would advertise more and create better docs, like using it with Apollo, not just being some page lost in their docs, for people who basically are trying to migrate to graphql and need to do it gradually.
NextJS and Gatsby are too opinionated/simplistic/magical and they’re no go for us. They’re coupled with view layers something Meteor did and learned not to do the hard way. Before Handlebars was popular, today (react/vue) and next it could be svelte, we already have people using Meteor with Svelte something you can’t do with NextJS or Gatsby.
Even folks using Meteor can’t pinpoint what Meteor is and who does it compare to, this is a real marketing identity problem. I personally see Meteor is the Ruby on Rails or Play Framework for NodeJS or as diaconutheodor put it: “the king for backends”. And I think it should be marketed as such, make it the go-to graphql backend server by shipping it with Apollo by default (with accounts) and easily pluggable view layers packages and market it as such.
6 Likes
Haha, maybe one day! Would really like someone now though…
A Galaxy/Apollo setup would make sense. They could even spin up a second instance for the apollo server, and optimize that separately. There are probably some great middleware they could produce to make something like user accounts work on Apollo, without or without DDP. There seems to be a last mile to complete to make Apollo work on Meteor in an easy Meteor way. (That could even be Meteor 2.0 - maybe they are driving there, who knows).
The way to balance Apollo with Meteor, is to think about Apollo as a high powered, very robust data tier, and Meteor as a somewhat opinionated, easy to get started, zero-config app platform. If we think about it this way, it can make sense to start to think about how Apollo could be integrated to replace DDP in ways that has some decisions made for the user by default (similar to how DDP already made those decisions).
On the broader conversation, I agree that WebPack and those others let you get lower level easier, but they also often require it, when updates happen that break everything, and that’s just not always productive. The nicest thing about those other build tools isn’t the tool itself, but the active communities around them. In some ways they have more up to date features (like WebPack offline) - but not always - Meteor’s modern bundle is further ahead than many of the alternatives, and there are still advantages to atmosphere that npm can’t reproduce. I try to produce some packages and PRs for Meteor to contribute to the ecosystem in my own way.
BTW, I’d be stoked to work on projects like these at MDG! I wonder if they have a position open. Thanks for the shout out. 
4 Likes
I think of developing an app as having 3 major concerns, how dev works, the web app stack, and hosting/scale.
Development:
- isomorphic - js on client and server. Even today I think of my mostly react projects this way. One of the things I love about React is that I can write my views once, and it works on both client and the server. I don’t think it’s important to have JS in the database. Sometimes I miss my SQL to be honest.
- Tooling - editors, debuggers, performance and code analysis, etc.
- The compiler/bundler is technical part of tooling, but it’s a big part, so it gets its own line.
- Atmosphere gets a shout out because it still does things npm can’t.
- zero-config, easy updates/backwards compat - this is HUGE. I HATE having to fix broken webpack scripts because something updated. The only similar problem I have in Meteor is abandoned community packages…
- local testing server
Web App Stack:
- database/data sources
- app server (optionally, an additional data server like Apollo, and other services like user accounts, image hosting, etc.)
- Data transport from app server to client. This can be DDP, or it can be Apollo (which actually can exist on a separate server, but it’s the same tier of the stack).
- Front end - this can be anything, but meteor is agnostic. Even here though Meteor offers some default sets - you can get up and running quickly with either Blaze or React for example.
Hosting/Scale
- Easy hosting options - Galaxy is one example.
- File hosting - Slingshot or something like it is necessary.
- Hooks for challenging scale points, like swapping Mongo for Redis Oplog (or however that works), or changing over to Apollo, and reducing the use of DDP.
So that’s a lot of stuff to juggle (and I didn’t list everything, no Cordova or realtime reactivity mentioned for example), and Meteor has good default decisions made for most of that. It used to have strong opinions on most of it, but has been removing some of those opinions, or simply making them (un)pluggable. When it became clear that React was a great way to develop front ends, they rightly opened the door to allow it to live along side Blaze (which also opened the door for Vue and Svelte). When it also became clear that Minimongo and DDP, while super easy to start and use, didn’t answer the hosting/scale question, they started to do the same there, while also capitalizing on an opportunity to use what they learned from that to produce a standalone data competitor which does answer the scale question, at the expense of simplicity (though I’m sure some packages could be created to help bridge that gap).
All of this gobbledygook is meant to address the question of “What is Meteor?” The answer for me is that it’s most of these things, but what do we call that? A great way to start and grow web apps. It is great developer tooling and it’s a great app stack with solid defaults. Are there similar platforms that offer as much out of the box?
The weak points are few - we could use some better tooling around performance and code analysis (specifically, APM, and a better bundle visualizer/analyzer) and the scale issue, but there we have Redis Oplog and Apollo for when the going gets tough.
7 Likes
This is something that has been really high on my wish list. Everyone once in awhile I try to dig through the fast render source to see what minimal source can be extracted to accomplish this, but much of it works goes waaay over my head and I can’t make sense of it 
Agree, this is good description of Meteor’s strength
For APM there’s nothing better than DataDog - which anyway is the essential tool for all logs, infrastructure monitoring and optimization of it.
As for the scaling issue you mentioned a couple of times, I haven’t run into that on MongoDb though I know from my own experience that it’s surely not the most performance NoSQL document oriented database out there (that is Couchbase for me and they also have an article with 3 main reasons why they are way superior in high throughput situations).
But the biggest weakness I see right now is the lack of upgrading MongoDb to the latest version along with changing to the latest npm driver! Which is just an example that is creating a PITA for us, other SW developers might have a different problem with Meteor not upgrading regularly anymore.
But like I wrote in the other thread, why should make MDG MongoDb better in Meteor when it’s trying to sell Apollo as the best solution?
Decoupling Meteor from MongoDb and giving easy access to any database out there (that fills the need of the developer) will help. But for that we don’t need Apollo, it’s an overkill vs adding other databases to Mongo/MiniMongo.
The advantage of Apollo is IMO in using multiple (different) databases in a very simple way (not saying zero configuration but yeah, a lot less painful).
So for us that might be a future path, using MongoDb or replace it with Couchbase if it becomes necessary, use Neo4j for all our graph-related data & graph queries and finally use either wide column store dB like Cassandra or a hierarchical SQL database like MariaDb for our genome data.