New package: mdg:validated-method - Meteor methods with better scoping, argument checking, and good defaults

  • Client and server execution differences.
  • When to use isSimulation vs isClient (I honestly have no idea!)
  • What (if any) interaction with deny/allow rules.
  • Isomorphic ID generation options and guarantees.

Most or all of these things are mentioned at the bottom of the outline. Have you read it?

The content can be seen on GitHub in the PR here: https://github.com/meteor/guide/pull/121/files

Not sure what happened there. When I followed the link the first time, the outline.md changes were not there (or I simply did not see them). Browser or brain failure. Sorry to waste your time!

Random idea: what if this package was published under the meteorguide:methods organization? That speaks more to its context etc

###Flux.

There’s been no mention of this here so far… In an isomorphic app using flux, you would want a store managing incoming actions, so they validate, mutate, etc. In the past I’ve had a meteor method invoke an action on a store after validating it, but given this new package (great job btw on 0.2.0 really appreciate the acute response to our feedback!) I think it might be a good idea to discuss any recommended patterns. Possibly even a little—supplementary—wrapper package/function/whatnot to cut down on boilerplate?

The guide mentions react a few times, and flux not at all. While those two technologies are independent of meteor, a huge chunk of the community uses them with meteor. Especially considering the new Blaze thing (not much interested in blaze, sorry lol) is supposedly going to be sitting atop react.

I’ve been using Alt—this isn’t specific to alt, but flux/redux/etc in general—If we can come up with a recommended Flux+Methods pattern, I’d be more than happy submit a boilerplate abstraction and/or contribute to that section of guide.

Interested in your thoughts on this.


Hmm, this is interesting. I could see the delineation between core packages on meteor:* and more-or-less optional packages under something separate being beneficial, but then again possibly bad due to less exposure/SEO. This could help cleanup the ./packages directory from the main meteor repo… I’ve never been a fan of having so many things cluttered in one place. (not that meteor should have a separate repo for each one of those packages lol)

I think the guide mentions flux kind of in secret. I noticed that it suggest getting data from a topmost template and pass it down as properties and contexts in contrary to direct data retrieval access from any template. Furthermore, it mentions in a few places that reactivedict and even local collections can be used to manage a state object store.

I think these are steps towards a flux architecture while keeping the terminology strictly within blaze, tracker and minimongo to avoid confusion, if that’s the intention.

2 Likes

@serkandurusoy, I think you’re on to something…

Based on Geoff Schmidt’s talk the other night, putting it mildly, it seems Meteor might one day soon closely resemble the Facebook (FB) stack (with the addition of DDP)

It doesn’t make sense to adopt React as the official view layer, but not all the underlying tech that FB builds around it, the tech that synergies and complements it.

@aadams my initial reaction to the react announcement was negative on the premise that enterprise software development does not take it too well when things don’t get multiple years of solid support and shifting gears too quickly might not always be a good thing.

Then there were talks about keeping Blaze as the officially endorsed layer while using react as the “rendering engine” for an upcoming blaze upgrade. I was sceptical about that as well.

But to be honest, seing that the guide is preparing us first with a migration of the mental model, showing that the current toolset can in fact be used in a way that resembles the best practices and patterns from a new technology, without changing anything other than how we (can better) architect out software - and that is a good thing - actually made me believe that MDG will deliver on the promise of a seamless transition.

Now at this point, if MDG can better communicate these developments and their vision in terms of actionable and tangible items that we can see and touch today, then I would not mind changing underlying technology (with the ability to do it at some point of my own chosing). I think that does actually deliver on the 7 principles and the original vision.

I also want to commend @sashko for his ability to (often) keep his cool under such heat and the efforts he is putting in transparency and leadership. One thing to add though, that perhaps the gems (in terms of communication) hidden in disparate places like multiple github issues and documents could be better promoted here on the forum as well, at least with some key takeaways. Afterall, github is for the progress while the forum is for communicating that.

3 Likes

This probably isn’t worth changing but maybe rename “_execute” to “test” or “callWithContext”? Does the underscore imply that it could change in the future? Also execute is pretty generic, isn’t obvious what it does.

1 Like

Just a question: Is this stable?

Hmm, I wouldn’t expect any super major changes, but I’d wait until Meteor 1.3 and the first version of the guide land to really rely on it. On the bright side, it’s so few lines of code that it wouldn’t be a big deal to fork it to your own version if you need to.

1 Like

Cool! So this would be part of 1.3 as well… npm loading and this? Anyway, I would look forward to using it. I guess for now I’ll stick with Meteor.methods

I’d say we consider this method package is a subset of the Meteor guide, which will be a really big part of the 1.3 release. We need to get some more feedback from production users about it before everything is finalized.

Hmmm… well I was hoping I could use it on my production project but it might have breaking changes if it switched to Meteor 1.3

Great to see some thought on how to use Meteor for large-scale applications!

Please forgive me for dumping some slightly unstructured thoughts here :smile:

CQRS

First let me suggest a perspective: looking at Command Query Responsibility Separation as an inspiration for API design for Meteor. I’m always kind of surprised that the term CQRS hardly ever shows up in discussions of architecture in Meteor, because Meteor is a pretty awesome modern CQRS framework. Let me briefly describe the important parallels:

A ‘traditional’ non-CQRS has the server going through ‘domain logic’ code for all its requests. An app with an Active Record style for example, has you using Post objects loaded with query logic in your update / create / remove stuff and it has you loading those same objects with update / create / remove logic even when you’re just displaying them in a list. CQRS tells us to separate our app logic into a command part and a query part. And guess what: Meteor methods are command handlers, and Meteor publications are a thin read layer. BOOM, you’ve got CQRS! But like a rocket-powered version, because the publications use livequery and get pushed from the server to the client for auto-updating goodness, oh my! (I often feel that an understanding of CQRS is why I’m often a lot less confused about Meteor’s architecture than a many of my colleagues) Also, MongoDB is an excellent database for denormalised data storage that goes very well with this architecture.

Now given that Meteor apps are basically CQRS apps, this means we can use lessons learned in the CQRS world to improve Meteor! Some examples:

Middleware

Given that Meteor Methods correspond to the POST / PUT / DELETE part of an application, it would be nice to use the middleware pattern to for purposes like logging, access control, maybe early validation.

Domain Driven Design + Optimistic UI

While CQRS is a natural fit with Domain Driven Design, the proposed validated-method makes you separate validation and behaviour, rather than coupling them like you would in a DDD app. I’m still searching for a pattern to couple this and still use Meteor’s client simulation.

Example: I have a form. Upon submission I would like to run domain code to perform domain behaviour and throw errors if necessary. Obviously I want all this to run on the server for security reasons, but for a quick response I also want to run it on the client. Now, on the client I want to catch potential errors from the simulation and the server, and use them to create user feedback on the form view. But oddly, when I listen for errors, Meteor decides to wait for the server response rather than trusting the simulation? From this point on I could just as well have left my domain code in /server instead of in /lib

The Meteor guide considers the optimal Meteor method to be one that waits for client validation until it goes to the server. However, for the applications I build, the correctness of my code (hence DDD) is more important than a bit of server resources.

In the guide, I see the usage of two options returnStubValue and throwStubExceptions, but I don’t see them in the Meteor documentation. They sound like they would allow me to make methods behave like I had expected their default behaviour to be. Are these 1.3 features or undocumented 1.2 features?

Persistence

I’ve learned that when working with CQRS, NoSQL and Meteor: it can really pay off to start thinking about persistence very differently from how I used to think about it when I still used web frameworks that prefer SQL. Especially when you start getting comfortable with data duplication, a lot of possibilities open up. A couple things I learned:

  • Database schemas should not handle all of your app’s validation, mainly because of transactions (looking at you, Collection2)
  • Denormalizing data is ok!
  • You don’t need an ORM
  • For some apps Event Sourcing is the way to go
  • Even though we use MongoDB: Make the schema explicit

I know that data duplication / denormalization feels really awkward for people coming from a SQL background; Which today is lots and lots of people. This is why you get the ‘Where is the ORM in Meteor?’ type questions. I found that as long as you make your schema explicit on the boundary between your domain and your storage, you’ll be fine. This boundary could be made with the Domain Event pattern or with the Repository pattern, for example.

A simple example of the repository:

var PostsCollection = new Mongo.Collection('posts');

PostsRepository = {
  findOne(id) {
    return PostsCollection.findOne(id);
  },

  insert({title, author, content}) {
    return PostsCollection.insert({title, author, content});
  }
}

This allows you to encapsulate the details of object persistence, and so I know exactly where in the code the database schema is made explicit.

Things that bug me about persistence in Meteor:

First, I can’t create the same collection in two places in my codebase, because Meteor will automatically try to create the /collection/insert etc. methods, and considers this a conflict.

Second, publications only take cursors. This means I often need to write two difference ‘findOne’ methods on my repository:

// lib
PostsRepository = {
  publishOne(id) {
    return PostsCollection.findOne(id);
  },

  findOne(id) {
    return this.publishOne(id).fetch()[0];
  },
}

// server
Meteor.publish('post', PostsRepository.publishOne);

// client
Template.blog.helpers({
  post(id) { return PostsRepository.findOne(id); }
});

Not sure if it’s possible to fix this, obviously the client needs to know which collection to add the document to…

Final thoughts

So ummm, concluding: I love Meteor’s low barrier to entry, and it’s brilliant pub/sub implementation. I’d love to see that in becoming more mature, Meteor will continue to play to it’s strengths, as it did before with using Cordova and MongoDB with isomorphic JavaScript. Thanks for all the great work on such a fantastic platform!

2 Likes

Yes, we could add a feature to ValidatedMethod to allow you to avoid passing those options. They are undocumented options that have existed for a while, presumably we should also document them.

This is not true - if the simulation throws an error, it immediately calls your client-side callback. That’s actually the main goal.

So do you want client-side errors to matter, or not? What should happen when the method simulation throws an error, in your view?

We do something similar, nice to see an official package.

foo: T.makeMethod('something.foo', function(bazz) {
  foo.validate.message(bazz);

  foo.authorUserId = Meteor.userId();

  return Bazz.mutate.bar(bazz, {fizz: 'something'});
}),

This has significantly less boilerplate but it seems ValidatedMethod does more. We wanted a simple wrapper.

This just defines the method and calls it. Doesn’t that already handle the stubs for client?

Then when we want direct server access we just call the mutator directly. The above would only be called for untrusted front-doors.

ValidatedMethod has more code because the above code seems to take advantage of foo.validate and Bazz.mutate.bar, which would both be part of the validated method definition when using this package.

1 Like

Yeah, I see the enforced structure and keeping it packaged all in one area. That’s a lot of boilerplate to assemble that but maybe method enforcement structure is a good idea. I’m not sure. So far doing what we have feels ok and if we need flexibility we have it too, but that can go either which way.

This is the source btw:

T.makeMethod = function(name, fn) {
  //install the Meteor.method
  Meteor.methods({[name]: fn});
  //return the Meteor.call wrapper
  return function() {
    var args = _.toArray(arguments);
    args.unshift(name);
    return Meteor.call.apply(Meteor, args);
  };
};

We became accustomed to defining our mutators and validators in their own objects.

We’re going to review ValidatedMethod and see the pros and cons of making the switch. As we’ve said before, we like to stick to the Meteor way ;).

1 Like