I dont get it; or the DAG of Meteor

help
1

A friend and coworker is absolutely besotted in Meteor and now we’re looking at using it for the next big project. I however don’t get it. It seems “fine” for minor projects; prototypes and throwaway apps; but not something you’d want to use for serious programming.

It seems like I’m missing something, and that makes me unhappy; and when I’m unhappy I notice every little annoying thing to the point where it seems like Meteor saw best practices and standard patterns and then deliberately set out to break them.

I don’t actually believe that of course, but since I don’t know what I don’t know it’s hard to ask for what it is.

It’s like the directed acyclic graph in git. When first I started learning about git, every tutorial mentioned that git uses a DAG, but that always seemed like an irrelevant implementation detail. Then one day I found some article that resonated, and suddenly everything made sense. You can plod along in git without thinking about the DAG, but everything gets much easier if you understand why it is important that git is a DAG.

I’m looking for the “DAG” of Meteor; the thing that makes everything else make sense. Does anyone know what it is?

3 Likes
2

You know that old saying

There are only two hard things in Computer Science: cache invalidation and naming things.

Meteor provides minimonogo and isomorphic APIs to address those things. Everything else follows those two things… ease of learning, rapid prototyping, reactive UIs, etc.

That’s my take anyway. $0.02

13 Likes
3

This is by far the most powerful pitch I’ve heard for Meteor!

4

As an end user and project manager, the DAG is:

Spend more time making the thing than making the thing that will let me make the thing.

This is a GREAT question - and I commend you, @professorphenomenal , for questioning without trollenging (trolling += challenging - you heard it here first). I’m glad folks like you are giving it a serious look and asking the head scratcher question, and hope we can give you a great answer.

@awatson1978 gave, of course, a fantastic answer. I’d pay attention - she was (is?) on the core team (I think?), so that’s probably about the best technical DAG you could ask for.

My answer comes from the end user perspective, and while the technicals are the foundation, I think the story is larger than that.

We helped DARPA build two games for its Crowd Sourced Formal Verification project, and we chose Meteor as the back end, way back at v0.5. We are continuing to use it for our gaming product back end, which we hope will scale to hundreds of thousands of MAU (knock on wood - or more!).

My decision was based on, in this order:

  • Solid team with backing == staying power
  • Full stack solution == not having to re-build bridges between API1, API2, API3…
  • Active Community, with ecosystem support from MDG == a suite of solutions I can install (a la npm), common problems are solved already, with less likelihood that I run into “Well, the fix worked in my setup, but might not in yours”
  • Isomorphic API (ok, I had to google that) == reduced dev and maintenance costs

Technically, I liked

  • Automatic reactivity w/ Collections
  • Data on the wire
  • Secure methods

I think Meteor is thriving in the community of people who want to get to MVP, get their app up, quickly. I see that as an audience that’s less concerned with ‘serious programming’ than ‘making it work’. I anticipate a lot of startups will encounter the “Oh, crap” moment when they go to scale, but that’s intrinsic to startups, not Meteor users. Sidebar. </soapbox>

So, back to my DAG - Meteor hits that sweet spot between helps me build the thing and isn’t a total hack and won’t need to be completely rewritten. It allows me to have both short-term and long-term wins.

Out of curiosity, how do you define ‘serious programming’, and what about Meteor precludes its use for same? You referenced best practices and patterns - is that the primary concern? Which one in particular, or what are the top three ‘violations’ you see?

Best,
Aaron
Whose words aren’t as efficient as others’, so he needs to give more for $0.02.

2 Likes
5

I have to say I’m a little surprised this hasn’t seen more action.

Is Abby’s response just that good?

Did it give you your DAG?

6

This, I think, is the root of my concern. I like “making the thing that makes the thing”. It’s the guys on the other side of the wall that “make the thing”. :smile:

Shifting that mindset is probably what I need; and useful too.

Going into specifics:

Clientside replication of a Mongo database, minimongo as awatson1978 called it; I include the DDP and similar too. This is an integral part of Meteor, but what if my data needs a relational database; or a graph database. Or as I’ve been leaning, something similar to Datomic (which makes cache invalidation seem like a quaint concept from a bygone age)?

I haven’t done my due dilligence on the datamodel yet though, so I really shouldn’t complain about it.

Isomorphic API is great. I am a bit miffed that Meteor-sites, in general, don’t seem to work without JavaScript and don’t even have the decency to include a noscript-tag apologizing for it. That’s just lazy.

I regret the “serious programming” comment. I should have said it doesn’t seem maintainable or usable in large projects.

Meteor seem well suited for a small team making a single product that will be complete in six months. Thus I reason, it will not be suitable for large teams making a big project that will live and be developed for ten years.

Mostly this impression comes from “throw everything into files, call them ‘whatever’ and let Meteor load everything”. A file anywhere that includes a body-tag will put stuff into the body. You export by making globals. Dependency Injection seems nonexistant.

As for violations of best practice; I did mention that I notice them because of a generic unhappiness, not because they are actual problems.

A few examples. Globals, id est declaring variables with var or not. Not using npm. Not using CommonJS or AMD lodaing of modules. Using mixins in React. State.

Aside, as an explanation for the late reply, I’ve been ill for a week.

7

And then they do what ? Rewrite with another framework ?

8

And then they do what ? Rewrite with another framework ?

To quote my college CS professor: “it depends”.

Again, that was a sidebar opinion of mine as I’ve perceived the Meteor early adopter community. I don’t think any Meteor-based startups have hit that pain point yet, which speaks to the OP’s concern - until someone’s been through the ringer, it’s hard to have confidence about how it will scale…until you’ve seen it scale.

Like, a bunch of times.

2 Likes
9

Welcome back! Glad you’re feeling better.

I’ll try to kick the can down the road, at least on some of the bigger items.

Your comment about relational databases has been a huge topic, and is becoming bigger all the time. Many of us have been agitating for alternate database support, and pointing out that if you want to go from ‘green’ to ‘brown’ (i.e. startup/new code to enterprise/existing code) customers and applications, you will NEED to integrate with other data stores. This is a huge blocker for Meteor’s market penetration, IMO. Since there was an architectural decision to marry up so closely with Mongo (especially the minimongo bit), replacing it is major surgery. I do know they’re working on this, though, so you might find that within some relatively short amount of time (on a planetary epoch scale ;)) you would have those options.

The noscript tag is a good point - about site developers. I know we’re guilty. It’s a poor assumption on our parts. I will say there are people working on SSR packages that could serve as a fallback or alternate solution somehow. shrug

All of which is a bit of a regrettable departure from the clarity of your original question - I liked that you were trying to find the “so what” of Meteor, and now we’ve gone down into the weeds.

As I step back and look at where this conversation has gone, I think maybe the DAG answer is buried in the question.

In other words, maybe you’re exactly right. Trying on a new thought, here…

Maybe Meteor really is best-suited for apps that will not likely scale beyond X users (simultaneous, MAU, whatevs) or Y team members or integration with Z external services, nor will it ever implement Favorite Features A, B, and/or C. I’ve found that every new programming language, framework, paradigm, etc. is a response to the state of the industry that birthed it, and Meteor is an attempt to say “Web pages are changing. They’re not the 1997 front page of Yahoo anymore. We need them to feel more like mobile apps”, and everything flows from that. No-one likes to talk about what you CAN’T build with a framework (especially when you’re building the framework), and I think that’s what you’re doing here, raising questions about how far it can go and what it can do. I think that’s great - you should know what a tool does before you use it.

So - if we start with the destination “framework for rapidly building responsive web apps that solve a single problem”, and derive from first principles, we can tick off the design decisions. Relational database. Required? Nope. Chuck it. Globals? Who cares - this is a single-page-web-app. Noscript? Well, then I’m sorry, it’s not me, it’s you, this isn’t going to work. Dependency injection? Meh. This app is a walled garden, and we’ve built a templating engine to handle management of the body. We know what’s going in and coming out, and we can always check your Meteor.userId().

Maybe the DAG is “No, right. Exactly. What you said.” :smile:

If you haven’t watched Geoff Schmidt’s talk from the August Devshop, I think he does a great job of encapsulating this. (In fact, I suspect I’m just parotting the Kool-Aid for you.)

Actually, now I’ve gone back and re-watched it - here’s the closest bit to a DAG that I could find:

“As .NET is to the Common Language Runtime, and as Java is to the JVM, Meteor hopes to be to Javascript.”

You really need to watch the whole talk to grok it - he really does do a great job of setting it up - but I’d say if you watch that and you still don’t buy it, or don’t see the value, then Meteor might not be meant to solve the problems you want to solve.

< /blovationMode>

10

I don’t quite get the scaling argument.

Any piece of (modern) technology today, and I mean it by any, is scalable enough by throwing more power, either in terms of single tower resorces, or cheap servers at it. Granted, some are (much) better than others.

The real scale issue comes in where the stack cannot benefit from computing resources. But then, that means the app has really grown. That also implies business value, with which comes either profit, or investment for future profit.

And when an app grows that much with a business model, there is no one single technology that can serve every part of it, thus one tends to engineer the beast from ground up, mixing and matching relevant technologies.

But the “seriousness” of an app does not necessarily correlate to its scale. I’ve delivered apps that handled millions of dollars worth of “serious” business operations which had just a few users.

Meteor’s got that kind of seriousness. You know that it works as it is advertised it should work. The beauty is the amount of work you need to get it to work. And the amount of work you need to keep it working. The isomorpishm, out of box features you get, the package ecosystem etc.

You also know that if and when it does not work as advertised, you’ll have access to both professional and community support. That is valuable. And the community is exceptionally vibrant to the extent I fear making MDG’s support services moot :wink:

11

Scaling is not just about raw performance / concurrent users, but also about scaling in complexity.
Who has implemented a real domain model in his Meteor app ?
AFAIK, traditionnal OOP doesn’t work well with the “automatic” reactivity of Meteor. Then you end up wiring everything by hand and there is no real point anymore in using Meteor.

12

Yes. I considered cutting the post after three paragraphs, but I had already written it… At least it gives some insight into my nitpicks.

I’ve noticed that too. Meteor feels like the first generation of a new way of building the web - when was the last time the first generation of anything was best?

If what you need to do fits well withing the previous paradigm, why not use a mature framework of that paradigm, rather than the first attempt at a new paradigm?

Of course, if the new paradigm is compelling enough there is only one choice.

Meteor feels very first generation, probably another part of my unease. I like Knockout, but it is really ugly and two way binding makes it complex. I wouldn’t introduce it to a project today, but back when we didn’t have Flow and React.

13

Who has implemented a real domain model in his Meteor app ?

OOP is a whole different topic and is certainly debatable.

Furthermore, even the most rooted stacks like Java have not figured out how to create and scale complex domain models, hence the existence of competing java frameworks such hibernate and spring.

So I think assessing meteor from an OOP perspective is unfair.

I, for one, did away with that and not looking back, even after almost15 years of oop-thinking in java, I was able to change my perspective and could not be happier :wink:

14

Here’s the catch, OP, Meteor was made to solve a specific set of problems, not become a monolithic framework for everything. Hell, even Node (that it was built on) was made to solve a subset of web development problems.

It’s about choosing the right tool for the job. The problem with software dev, sometimes, is that things become hyped.
You find yourself needing to hammer in some nails. You would choose the hammer for that job. If you choose another tool, it will just cause frustration and a terrible final product, obviously.

What meteor offers:

  • Publication and subscription made extremely easy
  • Data abstraction on the client and server
  • Rapid prototyping and building
  • Package based development
  • Abstraction from event loops
2 Likes
15

Hello. From the perspective of being the friend and co-worker mentioned in the original post, I must say that that list of offerings is the exact reason why I feel Meteor could fit in real good.

My main concerns are that:

  • We need more stuff than just those points.
  • We need a way to handle the the files that our system will create and make them reachable from with in 1-n instances of our application.
  • We have such a tight budget / deadline to get up to speed.
  • We actually don’t know how far we need to go with our products, or
  • if it needs to be as complex as it is today

These reasons both speaks for and against using meteor, which I’m well aware of.
But often I think that we as SW-Devs tend to add complexity that isn’t needed at all.

// Jonas

16

It’s built on Node, you can use fs

17

I know. :smile:

And there seems to be quite a few nice file handling packages as well.

18

This is a bigger issue though.

19

These reasons are also why I’m trying to get a more in dept talk with the Core team, so that we could dig deeper into the product that we’re talking about here.

20

While I agree with a lot of the questioning that has gone on here, my experience with SOA and enterprise software ecosystems is that we never think anymore of any one framework being the main choice for everything. Think of how fast the concept of “web frameworks” has grown over the past 10 years since the advent and success of Rails. But Rails itself is like an enterprise application, it can’t throw out its old paradigms and embrace new ones (like data over the wire, socket support, and other core tenants of Meteor) without alienating it’s user base. Certainly they try to add some new features every year, but they don’t think from a fresh perspective and perform a rewrite.

This is why so many Rails developers have moved on to either using Rails as an API layer only, to moving to other frameworks and languages. Today most serious apps that are “written to scale” are based on micro services, apps and message queues talking to each other and sharing events and data streams. This gets more complex at enterprise levels, where almost all the value of an application lies in the Business Intelligence and data mining areas, where trends and insights can be extracted from your user base so that a business can get key insights that will achieve more incremental growth, which is the nature of a business “at-scale”, not of one trying to gain traction and achieve product-market fit.

I think (wisely) MDG goes after the younger programmers and the startup communities, because in 10 years these will be the people making big decisions and scaling big enterprise applications. 10 years ago when I pitched my first Rails application, I never imagined that big companies would be trying to hire Rails developers to work on greenfield applications inside of a big billion-dollar business, but the reality of a big business is that they can’t build one monolithic application that manages everything, and have a team of 100 programmers all on the same code base, releasing code daily.

Meteor to me as a manager of a small programming team of 8, definitely hits the sweet spot where I can feel like it’s not just a toy that I am building. I can scale a meteor app far faster and easier with less money than my current Rails apps in production. I can hire junior programmers with JS experience and have them be productive in the isomorphic API without having to have them understand the MEAN stack (or worse), and I can easily find suitable programmers for the hard things (or do them myself). For me, picking Meteor as the next thing we are using to build another part of our SOA stack is not completely a no-brainer, I did wait almost 2 years from the point that I “fell in love” with Meteor, until today when we have official React support, SQL support on the horizon, and better professional deployment options.

Those are my 2$, I can’t find the cents symbol on my iPad.

2 Likes