Is it time to leave Meteor & MDG?

Planet Earth
62

QFT. If you’re getting to the point where you’re refactoring frameworks, you may find Nightwatch of interest. We’ve managed to use it to migrate Mongo/D3 applets from PHP to Spark to Blaze and now to React. Plus, they’ve got a solid PageObject pattern that can be used for React components. Also, it has the .meteor/nightwatch.json config file, which lets you configure it for any environment. As a test harness goes, it works fantastic for migrating apps through different frameworks, backends, and hosting infrastructures.

As far as the broader conversation goes, I’m sort of the same opinion as Arunoda… things will get refactored, Meteor will change, and it will continue to be Meteor. For those who are getting stressed about all the changes, consider this anecdote:

Meteor originally shipped with D3, and between Mongo/D3, was the only full-stack javascript data visualization platform on the market for probably 2 years. But D3 is popular in accademia, and not so many people use D3 in consumer apps; so MDG eventually moved it to community support in version 1.0. That sucked for some of us, and it felt liked Meteor stopped being the platform that we were originally sold on.

However; it turns out that there are a bunch of react-d3 packages, which have ported all of the D3 helper libraries (maths, stats, plotting, graphing, etc) and use React to manage drawing the SVG elements. ZOMG. Can we have some functional graphing in our functional programming UI? Best thing since sliced bread. And so it turns out that ditching Blaze and moving to NPM were necessary steps for us to do this super complicated refactor of D3 and get an improvement in the D3 rendering infrastructure.

So sometimes you don’t get what you want, but you get what you need. You just have to be prepared to refactor your framework.

tl;dr - Meteor is dead. Long live Meteor.

13 Likes
63

This is a good response. A few comments: I’m under-informed as I’ve backed off from Meteor until the dust settles. I was going to use it to teach 3rd year computer science students about the “write once, run anywhere” advantage of HTML/CSS/JS in Meteor and got halfway through developing a tutorial app. But the future of Meteor became too fluid to ask them to invest time in it right now, so I’m watching and waiting and using a.n.other framework, which doesn’t have that ‘run anywhere’ advantage built-in.

Good to see you’re not chasing trends; templating via Handlebars is well established (as Ember’s development of HTMLBars attests). I’m not a fan of adopting things just because large organisations (Google and Facebook) build them (look at Flash, which my uni stopped teaching only last year - big organisations take that long to change!), so one of the things that troubled me about the Meteor community was an over-emphasis on Angular and React when the JS community is much more varied and there are other significant players in the JS top 5. It reminds me how PHP became a no-brainer ‘default’ server language for so long. The herd approach stifles variety and competition and leads to serious lock-in which no-one sees at the time because “everyone’s using it”. Then it’s too late to change.

I like the idea of moving out and opening up towards the general JS community, and I think Meteor needs to be opinionated to a certain degree, but perhaps only about what it does best and not which third-party solutions to use as the default - it’s been described as a platform, not a framework, and that’s its strength.

2 Likes
64

Warning, this is strongly opinionated:

I tried Meteor in 2013. It was impressive but I thought it was way too messy and clunky for making anything serious out of it.

Then the whole blaze dying thing happened and the scary, foreign, react framework was the new favorite.
I asked the same question. “Will I use Meteor next time I need a web framework? Why learn some weird facebook thing if blaze works?”

Well I’m back coding Meteor again and OMG, react and Meteor is the light. It’s so vastly superior to blaze it’s a joke. And the learning curve is not steep at all. It’s actually pretty smooth and easy, if you’ve got a bit of versatile programming experience like node.js and you’re not a one trick dog.

MDG is rocking as always. They’re open about their intentions and have brought great tech to the open source community. FOR FREE. Be Grateful!!!

Trying to adopt and cling to software that’s not going to change or die means you should learn to program in COBOL. Other than that, you’re on your own.

So either roll up your big boy sleeves or eat dust.

To break it down for you, if you don’t like learning new things, either you can learn COBOL, or find a new profession.

7 Likes
65

if last paragrpah come true, that WILL BE the end of the Meteor stack (sadly + period)

66

Yes. 100%. Without a doubt.

I am so perplexed as to why Meteor insists on being its own monolithic framework, despite being built on Node and NPM. Its own build system, its own package system, its own installer? Why? Seems like a lot of bloat.

Everything should just be a small NPM package that’s independent. Take for example, minimongo. You should be able to download minimongo without any other big requirements. That means, not even a persistence layer on the back end, just minimongo.

67

You have to remember back to 2012 when it was first released… or even earlier to it’s inception. Modules were experimental and the norm was still to paste in a script tag into the bottom of an HTML page. NPM was not used for frontend and it was difficult to make it work for that purpose. There was no standard way to build files… just the early version of Grunt which makes webpack configs look tiny.

You also have to keep in mind that building a monolith offers greater engineering speed up front. This is key in a startup sometimes. Decoupling minimongo is non trivial if it could work with other databases… many have tried. This is why GraphQL clients are ideal… a structured way to query and mutate data regardless of what the backend is.

6 Likes
68

Actually there was a framework that was really ‘all that’ just before Meteor: Socketstream

Never got the OS traction it really deserved, outside closed circle of SV, faced similar community/tech strategy/directional issues… and Owen moved onto CTO at a big publisher, I believe

Now my faith is still with MDG to deliver on Apollo’s tremendous potential… thats currently more exciting than the front end imo.

69

In theory, it allows Meteor to be something more than NPM. To be that “.Net for Javascript” that MDG talked about for awhile. To be more than just web hosting. To include external pipelines, such as Cordova and Electron and XCode. To look at the entire Javascript ecosystem that extends beyond NPM and web hosting.

Unfortunately, the financial pressures to create an ROI on the VC investment appears to have put that original vision on the back burner. And if those external pipelines aren’t producing ROI, and the focus is on web-hosting as revenue-generating… then, yeah, why not just go with NPM?

I try to console myself by looking at Atmosphere as a refactor step… it’s allowed us to identify which features we want in clinical apps, and to collect and organize code which will eventually get migrated to NPM. For instance, we have something like 25 packages that integrate SimpleSchema and FHIR resources for interoperability with EMR venders. We know there are 100 total FHIR resources; and we now know how the packages should look like. So, when we migrate to NPM, we’re going to have a surgical precision understanding of what kind of packages we’re trying to create. So that’s something.

2 Likes
70

Here’s what bothers me – Meteor is just a framework for making JavaScript applications. JavaScript goes in, JavaScript comes out. It’s not a language or a compiler or anything.
Sure, it is quicker development up front, but in the long run for a bigger project, it becomes much harder to manage and development slows.

Given that’s the case, it should be no different than any other JavaScript tool. Why tightly couple the installer, build tool, and package system? If anything, that simply makes it completely incompatible with the rest of the JavaScript ecosystem.

So to make it less encapsulated, I think that these things should be removed and modularized. Say you remove the build tool. It might take slightly longer to perform a setup, but it would also mean you have more agency in how to build the Meteor project. Theoretically, you would be able to use Webpack with a Meteor plugin.

import WebpackMeteorBuild from 'webpack-meteor-build';

export default {
  /** ... */
  plugins: [
    new WebpackMeteorBuild({
      version: '1.4.0'
    })
  ]
}

Of course, if you want to use the default build tool, this would also be allowed easily. Maybe you could just do something like node -r ./node_modules/meteor/bin/build ./src/entry.js (similar to babel-register).

3 Likes
71

MDG claims they want to be like Microsoft’s .Net and Java of the JavaScript world right? They’ve given talks where they say this much (I can find a link if you like).

Has anyone in this room ever used .NET/C# or Java and SQL Server/Oracle databases? Have you ever used Microsoft’s web tech called ASP.net?

I have used .NET/C# and ASP.net web technologies and utilized SQL Sever as a backend; I started using it back in 2005. I have used this tech stack for at least 10 years.

My last role was as a Business Intelligence Developer using SSAS, SSRS, SSIS, and used ASP.net/C# as a front end to these services for an oil and gas co here in Houston. Even when my front end UI tech changed, like using on occasion winforms, I always knew how to get to the SQL backend and that I’d be using C#.

I can tell you that ASP.net/C# and SQL has changed only very little over the years, had a very clear upgrade path, and world class support behind it. I always knew exactly what the back end was going to be, how to connect to it to retrieve data, and how to build a web front end quickly. I can tell you any new position I applied for, I could demonstrate I knew SQL/C#/ASP.net and I was in the door. I can say this was true, from the start of my career to the time I stopped using this tech stack and I started my start-up with Meteor in December of 2014.

Here in Houston, where medical and the oil and gas industries are prominent, they almost all use Microsoft and the tech stack I described above. Having friends in other parts of the US, most all fortune 500 complies use the Microsoft tech stack, and some almost exclusively.

I’m not suggesting there’s a clear correlation between the predictability, stability, incremental change, large pool of developers that know the tech stack and become productive quickly because of it, great toolset, and proven support from Microsoft on one hand – and the adoption of this tech by corporations on the other. But it seems that way.

Since choosing Meteor for my tech stack in December of 2014 I’ve been please with Meteor and the tech stack behind it, including Blaze. Don’t listen to those that say Blaze is for demos or small projects. It’s served me and my clients well and performs wonderfully. And the reactivity provides something that I never had with ASP.net. In fact, that was one of the wow-factors that convinced me to give it a shot, as something like this would be very difficult in ASP.net. Granted, any one client doesn’t usually have more than 100 concurrent users.

But now 1.5 years later, after I have a running production application, the sands have shifted underneath me – to a degree I could not have imagined coming from a .Net background circa 2014.

I think all this talk about MDG’s fate and the fate of the Meteor/Blaze stack is simply due to the quick and total about face from MDG. Some existing devs may find these developments uncomfortable and bring about an air of uncertainty and doubt. In my case, I can only say right now its added risk to my company.

I think in the end it will be for the best to move to React/Redux and GraphQL and more importantly, away from Livedata/DDP/Tracker/Blaze. If we use MDG’s take/twist on the Facebook stack, I think this time we’ll likely see less change than what we’ve witness with MDG’s first crack at this.

In the long run, a year or two or three from now, their new strategy might pay off in an enterprise setting. They achieve their goals, both funding and recruitment. And we-all benefit by MDG then having the bandwidth to infuse the best ideas into Apollo stack (and hopefully with a few throwbacks from the original).

I expect MDG to continue providing a clear upgrade path to the Apollo world and hope their take on the Facebook stack brings along with it some of the great features that once wowed us with Meteor-original.

7 Likes
72

To be honest, I think a big part of this was the extreme acceleration in JavaScript developer tools in 2014-2015, including the entire React ecosystem basically coming out of nowhere. It quickly became clear that we couldn’t keep up with all of the developments in the wider community, and sticking to a 100% in-house stack would just leave the Meteor community falling behind the state of the art.

Today, I agree that things aren’t perfect - for example, the build tool could be faster, and it can be a bit awkward dealing with two package systems in the same app - but Meteor developers can now definitely take advantage of the overwhelming majority of cutting-edge JavaScript technology available today. Meteor is infinitely less a walled garden today than 1 year ago and more so 2 years ago, specifically because we felt it was necessary to enable the community to keep up with the breakneck speed of developments in JavaScript.

I don’t think Java or .NET were ever like this - there was never a completely new way to develop apps coming out every month, additions to the language, transpilers, etc. So while the goal at first was definitely to have that kind of support and stability, I think anyone can tell from looking at JavaScript as a language and community that it’s just not possible to have this at the moment.

13 Likes
73

Bye the way meteor was not the first framework, you had the mean stack but before that you had derbyjs which is what I started with and when meteor first came out it wasn’t open they changed that right away.

74

I thought after the change in strategy announced in Dec 15, one of the goals was to be the Microsoft/Java of JavaScript (something I’d never heard before that speech). And I thought this tied nicely into MDG’s enterprise strategy.

The move to put MDG’s stamp on the the Facebook stack also make sense. This adds name recognition and credibility, legions of developers that know React, and some sense of stability and support. Entering the enterprise (even medium sized) will require MDG to slow the pace of change and provide LTS on their customizations no doubt.

Side Note: I do appreciate implicit support to existing devs MDG has so far provided by way of the upgrade path. I only wish the deployment tools a lot of us use/used didn’t all break along the way (mup/mupx).

3 Likes
75

Lots of great comments. I agree that the refactor is probably necessary; and once the dust settles things will pay off.

This is all actually very reminiscent of jumps and hurdles people were going through with X Window Managers during the 90s… AfterStep, WindowMaker, Enlightenment, Gnome. Although with quite a bit more noise since so many more people use computers nowdays. But the tech infrastructure is following the same evolution as people are sorting out networking, styling, components, testing, etc in a browser environment.

I have two things that I hope MDG will do that will bring things back round full circle, and show a sort of multi-year follow-through on their original vision:

  • First, I hope that the GraphQL queries on Apollo can be run on the server just like they are from the client. Should be trivial enough to do some sort of loopback from the server. But the point is to keep the query language isomorphic between server and client. If that happens, and the query library is portable enough, we should also be able to port it to the Mongo shell as an extension, since Mongo shell is based on the JVM. And then we get to keep an isomorphic data query API across server, client, and database.

  • Second, I hope they provide an API veneer across the entire Apollo Stack, Saturn, Meteor+, and whatever other products they’re shipping that’s backwards-compatible. The Redux/Flux/Reflux/Flerux/Flrux is bordering on nonsense, and anonymous pure-function containers and components might be clever programming tricks, but they don’t make for a good API. So, I hope that once the dust settles, the Meteor API can get layered over GraphQL and Apollo.

6 Likes
76

Meteor is “mostly” open source, where’s source for the package server, the warehouse and build farm? MDG provides an array of internal Meteor services which we have no source code for… Will I really have to reserve engineer them? I mean, we don’t even have mirrors for the services, and I’ve seen them down before… Call a spade a spade.

1 Like
77

True, hence our effort over the past 6 months to decouple and “de-monolith” Meteor. To be fair, in the context of the response to the question about “what happens if MDG is acquired or shuts down”, I was referring to what’s currently in Meteor core.

1 Like
78

Totally agree. I was mainly thinking about coupling to databases but having so tightly coupled tooling is not ideal. On the bright side it seems like the move to NPM is going to decouple it slowly.

79

I haven’t tried the apollo-client yet on the server but i’m 99% sure the core API will work on the server as long as the fetch pollyfill is available.

I’ve been using a similar setup in Elixir with a GraphQL API that feeds to several microservices. All data mutations happen on the main API app and the services all use the GraphQL query/mutation to handle data. There is a private schema that is used that extends a few things like allowing the consumer to return the password field on the user object and things like that, but same interface on the client and server.

It’s a bit verbose compared to an ORM but it’s very easy to make a small wrapper for each module if you need a more compact interface. You can also change the transport layer for other server/server communication if http is too slow.


The Redux/Flux/Reflux/Flerux/Flrux is bordering on nonsense, and anonymous pure-function containers and components might be clever programming tricks, but they don’t make for a good API. So, I hope that once the dust settles, the Meteor API can get layered over GraphQL and Apollo.

Agreed. It’s getting ridiculous. However, the community has finally relied on Redux as the gold standard so at least there’s that.

If you opt into the apollo-react module to get Relay-like queries/fragments then i’ve noticed that I hardly use Redux at all… it’s mainly for truly global things like modals, drop-down banners, etc…

By using a container and pure component you come really really close to having a Blaze template and script file. You can use local ‘template state’ 90% of the time for things not saved to the DB, and Apollo handles the state for server state. I found that this eliminates a lot of the Redux ceremony that I disliked and makes things more predictable and more cohesive.


Example: 
components/
   posts/
     index.js
     posts.js
     posts.test.js   test template logic outside of data fetching in index.js!

// app/components/posts/index.js
import Posts from "./posts";


const Container = React.createClass({
  getInitialState: () => ({   // use local state as much as possible and pass to child template
  }),

  componentWillMount() {
    analytics.track("user viewed posts");
  },

  handleSubmit() {
    analytics.track("user clicked add comment");
    // do stuff to submit data, maybe call mutation?
  },

  render() {
    return (
      <PostsShow
         posts={this.props.data.newPosts}
         isPostsLoading={this.props.data.loading}
         handleSubmit={this.handleSubmit}
         />
      );
  }
});

export default connect({mapQueriesToProps: ({state}) => ({
  data: {
    query: gql` query _($userId: String!) {
      newPosts(userId: $userId) {
        id, title, description
      }
    }`,
    variables: {
      userId: state.viewer.id
    },
  }
})})(Container);

// app/components/posts/posts.js

const Posts = (props) => {
  if (props.isPostsLoading) return <LoadingView />

  return (
    <div style={styles.container}>
      {
        props.posts.map((post, i) => {
          return (<Post {...post} key={i} />);
        })
      }
    </div>
  );
};

Posts.propTypes = {
  isLoadingPosts: React.PropTypes.bool.isRequired,
  handleSubmit: React.PropTypes.function.isRequired,
};
export default Posts;


// partials

const Post = (props) => {
  return (
    <div style={styles.postContainer}>
      <h3>{props.title}</h3>
      <p>{props.description}</p>
    </div>
  );
};
Post.propTypes = {
  title: React.PropTypes.string.isRequired,
  description: React.PropTypes.string.isRequired,
};



const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: "blue",
  },
  postContainer: {
    height: 20,
  },
});
3 Likes
80

Somewhat related to this…

My question is what happens going forward if they don’t have that lock-in factor?

If you can use apollo straight from NPM without the need for the meteor build system, what happens to Galaxy?

With things like Create React App and React Boilerplate, along with the dust settling around web-dev-server/webpack/babel…which means configuring a build system will get less and less of a pain point and less of a reason to use meteor…

Where does it leave Galaxy and MDG?

The only real lock-in factors for using meteor (and thus using Galaxy) at that point are the account system and reactivity. If MDG isn’t going to monetize accounts like auth0, or focus on pushing meteor accounts, then people will just use auth0. Then we’re just left with reactivity, which sooner or later there will be an easy-enough substitute.

which leaves us with little to no reason to use meteor. which leaves us with very little to no reason to use galaxy (can you even use galaxy with a vanilla node app?).

I may be out of the loop on the stratie, but I’m not sure where MDG income is going to come from in 2 years without some sort of lock-in factor. How can you monetize Apollo?

Not to be too nosey. Is the goal to just build apollo and sell yourself into Microsoft to give them a javascript play on top of their new open-source play?

2 Likes
81

If Meteor’s business model relies on some kind of “lock-in” then I don’t think we are going to do too well long term. People use Galaxy because it works better for them than other hosting options, not because there isn’t any other way to host Meteor apps (in fact there are several that are constantly advertised here on the forums, like NodeChef).

At the end of the day, Meteor is a tool to create a modern JavaScript application, which pioneered a lot of the most popular approaches people use today. These applications are going to come with some difficulties, for example the need to manage persistent websocket connections. This is where Galaxy comes in - it’s the best way to host a stateful webapp with a peristent connection. That technology is just as useful regardless of what build tool, view layer, or accounts system you’re using.

For Apollo, the core technology is GraphQL. We think that anyone who builds stuff using GraphQL is going to need some GraphQL-specific tools to manage their app. So at the end of the day it’s not about locking people in - it’s about building great new tools that people can use to improve their workflow, because modern apps have more complicated architectural requirements.

One of the reasons we are focusing on the data layer is because it’s the place where there is the most progress to be made. Another is because that’s where people need production tools to host their apps, inspect performance, debug data flow, and more.

That’s definitely not our goal. Like I said before, if I wanted to work at Facebook or Microsoft I would just apply for a job there.

11 Likes