The future of Meteor

[imkost]

When Meteor was just started in 2012, there were no powerful build-tools (Grunt was just started), npm packages were meant to be used only inside Node and live reloading was a WOW feature.

But now, 3 years later, we have Webpack, which is a powerful build tool; npm packages now can be used for browser too; and hot reloading now looks like something old and slow since we have hot-code-pushes.

Seems like what Meteor invented when it was just started, now is implemented via other tools. And these tools are more focused, powerful and generally-applied (most of Meteor tools are just Meteor-applied).

I see projects popping up, which use Webpack to build Meteor projects. Webpack gives an ability to use es6 import/export functionality and hot-code-pushes for react components.

Meteor 1.2 provides new Build Plugin API, but do we really need it? Build Plugin API allows us to write compilers (es6 to es5), linters (jshint) and other stuff which are already implemented and easy-to-use via Webpack. So why don’t just use Webpack?

Web development changed a lot since Meteor started. Maybe it’s time for Meteor to rethink some of it’s fundamentals?

Questions that bother me:

1. Do we need Meteor packages? Couldn’t we just switch to using npm?
2. Do we need isobuild? Couldn’t we just use webpack?

Wouldn’t it be better for Meteor to be a bunch of npm packages which can be used together and combined with other packages?

Like this:

meteor-server
meteor-mongo
meteor-tracker
meteor-react // react bindings for meteor
react
react-router // not meteor-react-router which wraps react-router
momentjs     // not meteor-moment which wraps momentjs
+ webpack-config

What do you think of the future of Meteor? What does MDG think of this? Do questions above bother you too?

[mattkrick]

Solid suggestions, and these questions have existed for awhile, too. When meteor started, folks still had to worry about ES5 syntax (and no one knew javascript would be updated annually!) and the full-stack javascript framework was nearly impossible to get a handle on. You got builders and package managers and front-end frameworks and no one is going to tell you which technology in on the way out because that means they’d have to learn a new one (just go ahead & ask if django or rails is dying on quora, see what happens ;)).

For me, I think webpack is the future for large meteor applications and I’m in the process of switching an app over. But, for proof of concepts, MVPs, classrooms, and the like, the isomorphic setup is a pure joy to work with. You remember your first day with Meteor where in 1 hour you had a todo list app on your freaking phone?! That was magic!

The separate package manager has always been a pain point since the beginning, but it sure beat crossing your fingers & installing an npm package, which was the alternative (does it work with fibers, the current version of node, etc, etc). Not to mention NPM was (and still kinda is) a mess. Now that things are settling down (webpack is winning an audience, ES2015 has modules, node is at v4.0.0, native promises will soon replace fibers) I think a lot more becomes possible, and a ES2015 base is the first step. Even from their early talks, MDG always said something along the lines of Meteor not being a single technology, but a bunch of really smart packages that fit well together. To me, Meteor is a super well tested framework with fibers, minimongo, pub/sub, and websockets. Why someone would want to reinvent these wheels for each project when they could just use Meteor is a mystery to me. Lots of folks think Meteor = Blaze but as we move away from that thinking (and MDG finishes decoupling Blaze), a lot of neat things open up where you can trade simplicity for power. Sure, it’ll take some time & it’ll be a little painful for the first movers, but I think it’s headed in the right direction.

[timw]

I’d love to see Arunoda’s and Josh Owen’s responses to this.

[arunoda]

I can see there is a lot of tools coming in this field. I personally think Meteor could have been innovate more and focus on what it does best.
We could have focus more about DDP.

Now, Meteor started to follow the trend and starting to use other tools like React/Angular etc. That’s something good.

I think Meteor’s packing system is something great because of it’s single loading nature. All the packages were built on that concept and that specially helps client side apps. Yeah! NPM will catch up on that soon.

I know there are more tools like WebPack and others. What Meteor makes special is because it’s a well engineered solution. So, anyone can focus on the app, not trying to keep up with the integrations. I think MDG is looking for enterprises to join Meteor when enterprises started to build apps on JavaScript. So, it’s not too late for MDG.

That being said, there are a lot of stuff happening outside of Meteor. I could see that Meteor will pick the best stuff on the community(outside of Meteor) and wrap it nicely and distribute. That’s something good.

But, I really fear about the Meteor community. There are very minimal original development in the community. If there were some they are not active much. I mean, trying out new concepts and ideas and make the platform in a better level. That’s again a shortcoming of the community as well. They(we) don’t trust the community and always looks for core packages.

For an example, there is a cool project called anydb where it allows you to connect any database with Meteor. Yeah! It lacks some stuff, but it’s a promising solution. We should have promote it more. That’s a one project, there are a lot in the community.

I feel that MDG is not listening well to community as well. Sashko is doing a pretty good job. But it should come from the top and all of the devs.

If I were a Founder of MDG, I have done these things in 2014

1. Release Galaxy very early and test it with the community. No need to wait for Kubernetes.
2. Focus more on DDP. That means, there will be a lot more apps on DDP. Galaxy will have first class support for DDP. So, we’ve many apps in Galaxy.
3. Keep the core lean. Push most of the packages to Atmosphere.
4. Build a community leadership team and host 2 meetings at least a month.
5. Start a community fund which helps and motivate developers work on OpenSource projects (something like GitTip/GSOC)

[pal]

I’d like to add my unqualified tuppence worth

Meteor seems in some respects overtaken by the rapidity of WebDev. I assume Meteors trajectory is driven by very understandable commercial concerns: anyone here recall socketstream? Difficult to remain viable when you become an npm wrapper/DDP library/build tool.
However, thats what it becomes once devs start tinkering - bringing their favourite tools in, or chasing the next hot shiny thing -> and this perhaps is Arunoda’s point (?), theres more tinkering than innovation on the platform per se, from community and core

That said…its an ideal platform for the budding webdev and small teams…Can Meteor straddle the starter and pro boats? Big challenge. I’d guess a community/commercial editions. Looking at the talent there, id assume they’re well ahead on this.

[deanius]

Good points. Let me fill my glass, and wax rhapsodic…

I am very smitten by Meteor still, and many of your points don’t dissuade me because they would require several years’ experience in Web Dev to even understand the advantage over what Meteor offers. I’ve spent half a day setting up, then tweaking Grunt, wringing my hands over whether Broccoli is better, etc… Meteor gives them the way to bypass all that bike-shedding.

Meteor goes after neophytes, and ropes them in with simplicity. For people focused on their product, like those I see coming to my meetup, that’s the competitive advantage. It may take other companies several years to get the momentum and ease of tooling Meteor already has.

Still - there are things that concern me about Meteor’s future: governance and technology.

Governance: I’m less qualified to speak about than tech, but here are my points. Keeping Galaxy shrouded in dark matter for so long has not been encouraging, and gives me less to evangelize about. Even a partially-featured Galaxy would be better than “wait 20 seconds” meteor.com hosting. I don’t know why they keep Galaxy so… Nebulous…

Also, I’ve heard of PRs and job applications from people piling up at MDG’s door with hardly an acknowledgement. What kind of company can afford to turn away test-passing PRs or motivated applicants ? Hopefully one whose vision is very right and can afford to be so focused.

Technology: Here it is, my deepest fear - React may subsume Meteor, and that’s being optimistic compared to being totally sidelined. DDP is not a trade secret, and websockets are getting more ubiquitous. ES6 will force innovation in build tools and modules all around. Meteor’s deploy-via-Cordova advantage is lessened with the release of React Native. Reactive Programming is awesome indeed, but more than just we Meteorites are catching on. Blaze seems to have served its purpose, and if anyone else (FireBase, ReThink) can solve the reactive-db-resultset problem faster then MDG, that’s the nail in Meteor’s coffin.

The metaphor of the dinosaurs being killed by a single Meteor is an evocative one. But how do we know a smaller meteor didn’t fall a few days earlier? And are we Meteorites so sure ours is the only one that’s coming ?

Of course, I do love Meteor. Its gravity still attracts me. But I’d be happy to slingshot around it if something bigger proved to take me to a higher orbit

[supermario]

Re: meetups, over the last year or so I’ve been to a bunch of Meteor NY meetups, as well as events put on by Nodeschool NY.

As a Meteor fan, it gives me no pleasure to say that the Meteor events don’t compare favorably in terms of turnout, energy, and dynamism. I think @arunoda is on the money with points 1, 4, and 5.

[arunoda]

Hi,

I agree meteor puts a lot of effort on events and I never seen something like that in other place.

I love meteor and a die hard fan. Not just because we do Kadira, but as a platform I like. Events is just one side of the community. That’s for getting attention and trying make awareness.

But I was talking about the community product/feature development. For an example, see the community behind react. It’s volunteer basis. We could have done more to empower the community. And it’s not too late.

About the Galaxy and DDP stuff. It’s money related, but it’s not the all. Building a good platform is hard. We the community could help MDG to test it to out and fine tune it.

And we could focus more about DDP and there will be more apps. Not also in JS. Because it’s hard to build an app with a single codebase in long run. But I don’t see that with DDP. We have DDP clients. But needs more focus. That’s where we can help.

[joshowens]

I feel that MDG is not listening well to community as well. Sashko is doing a pretty good job. But it should come from the top and all of the devs.

If I were a Founder of MDG, I have done these things in 2014

1. Release Galaxy very early and test it with the community. No need to wait for Kubernetes. 2) Focus more on DDP. That means, there will be a lot more apps on DDP. Galaxy will have first class support for DDP. So, we’ve many apps in Galaxy.3) Keep the core lean. Push most of the packages to Atmosphere.4) Build a community leadership team and host 2 meetings at least a month.5) Start a community fund which helps and motivate developers work on OpenSource projects (something like GitTip/GSOC)

I agree with @arunoda on all of the above. I was part of a group called Vanguard, but that seems to have fallen to the back burner. I would love to see Meteor spend a little time on helping run a Community Leadership group that lives outside of the MDG - perhaps something different than Vanguard was.

I also think Arunoda nailed it on the head when he said too many people are waiting on MDG to build out packages, there have been plenty of people trying to solve the SQL driver issues, why does that solution have to be built by people like @sashko and @slava?

I think part of the reason people wait is that changes need to be made in core packages like accounts-base, and I haven’t found a good way to get pull requests accepted from MDG yet, my last fully tested Blaze PR has just been sitting there since May. I think there is a delicate balance that needs to be struck between what we wait on MDG for and how to work with MDG to make better solutions.

MDG will never be able to supply the kind of ‘velocity’ this project needs on their own, we need to figure out how to get more of us into the game and make something that is viable in the long-term in this ever-changing landscape.

[benstr]

Great and honest comments on this post. A lot to digest and think about.

[awatson1978]

Speaking for Clinical Meteor and the healthcare industry users, there’s a growing group of us (mostly behind closed doors, unfortunately; because of NDA agreements and liability concerns), who are getting ready to make an industry specific version of Meteor. When Meteor first came out, it had a Cathedral like approach on how to structure a reactive app; which has been slowly watered down as MDG expands its user base and moves towards a Bazaar model.

For what it’s worth, some of us are preparing to (hopefully) swing the pendulum back towards a Cathedral approach for an industry specific stack.

We’ve got backing from universities, corporate clients, an NIH funded project, a couple of consortiums, etc. And we’re probably going to double the number of packages in the ‘core’ of the new track, and include a slew of new technologies… barcode readers, radiology viewers, hl7 interfaces, ICD10 catalog, haptic gesture interfaces, near field communications, resource scheduling, form builders, data pipelines, scheduling queues, genetic algorithms, neural nets, machine learning, telemedicine videos, medical illustrations, medical icons, etc. etc.

Interesting thing to note: we’re doubling down on the original Mac/Mongo/Blaze architecture, and making no promises regarding Windows, SQL, or Angular. We’re going vertical with the stack, not horizontal.

As for the meteor tool itself, we’re planning on adding testing, scaffolding, refactoring, and FDA auditing tools; and are expanding the isomorphic APIs to include the testing environment and editor. We’re also talking with ex-XCode devs on tooling architecture and how to build an IDE.

Maybe our release track will give others an example of how the community can extent Meteor and take things to new levels? I can certainly imagine there being a lot of release distros… a security track, a react track, a bootstrap track, a social-networking track, a geocoding/mapping track, a robotics track, etc. etc.

Interestingly, the healthcare industry has like 40 years of using a single-language stack based on a document oriented database (M/MUMPS). So for our industry, it’s not an ever-changing landscape. The landscape is fairly well defined; but MDG only finished 20% of the stack, because it’s not a healthcare company. We see the strategic value of Meteor in terms of translating all these non-javascript things into javascript. Blaze vs React vs Angular is splitting hairs, and getting caught up in distractions from the long-term goal of translating a zillion legacy systems into a standardized single-language platform (i.e. javascript).

So, you’ll be seeing us go more vertical with the stack.

[miningsam]

Meteor 1.2 provides new Build Plugin API, but do we really need it?

That’s a good question. I would love to see the motivation (and especially the arguments behind that one)

Do we need Meteor packages? Couldn’t we just switch to using npm?

I personally am pro-Meteor packages. Targeted code vs generic code.

Do we need isobuild? Couldn’t we just use webpack?

… I am reading ‘do’ we need a lot here (sorry, my opinion only). What I can say that we all have different needs. And the MDG just can’t do it all at the same time. Hence there is a roadmap (somewhere on Trello, last time I checked, with a community backlog). Do we really need SQL support? I certainly don’t. I prefer more DDP client support. But, I can image a stormload of Heroku unsung heroes do.

The future of Meteor is bright. We still have some years until the next big mind blowing thing

[sacha]

Blaze vs React vs Angular is splitting hairs, and getting caught up in distractions from the long-term goal of translating a zillion legacy systems into a standardized single-language platform (i.e. javascript).

I think that’s an important point. While we’re debating Facebook’s latest shiny release, the vast majority of devs out there are still using PHP and WordPress. What matters to them is something that just works out of the box, whether or not it uses the latest technologies. That market couldn’t care less about Blaze vs React or Mongo vs MySQL.

So I think that while it’s important to stay up to date and be forward-thinking, it’s also not the only factor to consider. For example, if you’re looking through the “average developer” lens, you can argue Blaze is a better solution than React just because of its easier learning curve.

[teslan]

I am 55+ and quite new to web-dev or open source for that matter. The deeper that I get into anything the more I end up not liking and finding huge faults with open source projects which are primarily backed by for-profit companies. The more that I get to know about Meteor (framework and community) the more that it reminds me of 4/gl languages with which many people who should not be coding got into coding. This is backed by what seems to be the biggest disparity - where Meteor is so big and yet the community involvement with development of the core is so little. It mind as well not be an open source project.

[brajt]

One of the things I find at fault when we talk about community based packages vs core packages is an unfortunate, though reasonable tendention of tutorial writers to stick to core packages, if we don’t count angular/react flavours of simple todo tutorial.

I’ve yet have to see a tutorial that combines let’s say jade, coffeescript, blaze/flow components, collection2/astronomy + whatever you can think of, to show people how it really looks like and what’s possible when all of these is mixed together.

How are we supposed to mix such features if all tutorials are trying to stay away from them, to the point of often sticking to Sessions instead of Reactive Vars/Dicts/Classes.

[joshowens]

I’ve yet have to see a tutorial that combines let’s say jade, coffeescript, blaze/flow components, collection2/astronomy

I’ll just be honest, I write about the things that I use. Coffeescript is pretty much dead and I see no reason for it to hang around with ES2015 being out now with easy things like Babel to help get it working everywhere now. I did post about collection-helpers because I use that a lot.

I think there are a bunch of great points above from @deanius, @awatson1978, and @sacha. I came to Meteor because I felt like it sped up my time to develop an app to production by around 40%. I still feel that is accurate and true, today.

[acemtp]

I cannot agree more with what is said.

What I’m really frightened is to see the Meteor community split.

Before, everybody works on the same direction, blaze+mongo…

Before, everybody worked with iron-router. Today, we see the first split with iron-router and flow-router. Now we have some cool package that work on iron but not a flow and other on flow and not iron. Not a big deal but it’s the beginning.

Soon we’ll have Blaze/React/Angular so packages will have to make a choice. I really don’t believe that maintainers will often maintain the 3 versions of his package.

Soon we’ll have Mongo/sql, same problem.

We’ll have package that will work with sql+react+flow, some other with mongo+angular+iron, and so on.

How we’ll be able to handle all these options in packages?
Big fat packages that handle 3 templating + 2 db + 2 router? I don’t believe in that.
Some intermediate packages that will be isomorphic (like the one we can see for router). I doubt.

It can really split the quite small community in 10 smaller ones and it’ll sucks a lot!

[massimosgrelli]

So I think that while it’s important to stay up to date and be forward-thinking, it’s also not the only factor to consider. For example, if you’re looking through the “average developer” lens, you can argue Blaze is a better solution than React just because of its easier learning curve.

I completely agree with you on this. The question is who’s the target user/developer for a technology like Meteor? What’s the target project size? What are target companies?

We have to consider that, even if Meteor is just 3 years old, MDG had a lot of cash in the bank (compared to Rails for example) to build the core product and promote a rock solid community. Rails got pretty soon recognized as the framework people were using when creating a new startup. And that happened even if it had a lot of problems at the beginning. So in its case startups were the target companies. In the case of Meteor is not clear yet what is the “target customer”, the one with a perfect fit with its development model.

I love spending my days (and nights) coding with Meteor and I’m sure that the community will become stronger and stronger. But more evangelism and proactive marketing of this wonderful technology have to be done pretty quickly and IMHO it should be driven by MDG. They have the money and the engineering talent to make that happen.
For example, we’d need an MDG + O’Reilly (?) technology conference organized soon. We’d need people like Martin Fowler or Kent Beck involved in evangelizing Meteor beauty.

[niallobrien]

You’ll never see Martin Fowler evangelising Meteor.

[SkinnyGeek1010]

We’d need people like Martin Fowler or Kent Beck involved in evangelizing Meteor beauty.

It’s going to be a long time before that happens. There are no Meteor specific architectures (unfortunately)… React in Meteor is the first glimmer of template level structure and composition. Everything is geared towards making a todo app with as little code as possible, not how to make a maintainable app.

They would laugh at the forced global variables . I don’t understand how Meteor expects to get revenue from large companies & enterprises when their app can’t handle enterprise size app complexity. It’s like they’re targeting newbies (which is not a bad thing) with easy defaults and little configurability but they want Meteor to be in the enterprise.

Perhaps this will be resolved in the future.