New package - jam:method - An easy way to create Meteor methods

[ceigey]

It looks like there’s some API overlap with zodern:relay, but I’m assuming the tradeoff here is that jam:method can of course be used for optimistic UI with minimongo, where as zodern:relay does some magical code splitting behind the scenes so the client-side only gets a typesafe wrapper around Meteor.callAsync.

Regarding jam:easy-schema, is it worth name-spacing the monkey patched method attachSchema a bit by calling it attachEasySchema to avoid a collision?

Likewise, does the schema you pass into a jam:method constructor have to use <Collection>.schema, or can you call it like this?

import { createMethod } from 'meteor/jam:method'; // can import { Methods } from 'meteor/jam:method' instead and use Methods.create if you prefer

export const create = createMethod({
  name: 'todos.create',
  schema: { text: String },
  run({ text }) {
    const todo = {
      text,
      done: false,
      createdAt: new Date(),
      authorId: Meteor.userId(), // can also use this.userId instead of Meteor.userId()
    }
    const todoId = Todos.insert(todo);
    return todoId;
  }
});

I like the idea of having a “standard” based on Meteor check for validation, separate from solutions like Zod, Typebox etc (personally I am using Typebox in a meteor project at the moment, though we barely take advantage of Meteor’s features in that project until 3.0 settles a bit and I know what will be “safe” to double down on).

It looks like easy-schema can use “raw” schemas like { text: String } but it does some processing each time you pass in a “raw” schema, to make the schema into the right “shape”. So then I wonder if it’s worth exposing shapeSchema somehow to let people “compile” the schemas e.g. as a module export.

(I have more thoughts regarding easy-schema than jam:method at the moment, apologies!)

[jam]

Thanks for the ideas and feedback. I really appreciate it!

It looks like there’s some API overlap with zodern:relay

Yep, I took inspiration from zodern:relay, Validated Method, my own experiences using basic Meteor.methods with check, and reading a bunch of old discussions on ways to make methods better.

I like writing isomorphically and taking advantage of Optimistic UI — one of Meteor’s best features imo. So jam:method seeks to be a worthy Validated Method successor.

I think zodern does great work and like using the zodern:melte package and Monti APM. I truly appreciate all of their contributions to Meteor core as well. I think zodern:relay is a really nice package. When I tried it out, there were some things I wish it had which inspired jam:method. I also wasn’t sure I wanted to be locked into zod and the other tradeoffs it makes.

Regarding jam:easy-schema, is it worth name-spacing the monkey patched method attachSchema a bit by calling it attachEasySchema to avoid a collision?

I think the only package that uses attachSchema is aldeed:collection2. If you switch to jam:easy-schema, you won’t need aldeed:collection2 because it has that functionality built in. I wanted to have a familiar behavior for people who choose to switch. I suppose I’d be surprised if there would be a collision — seems like you’d either use jam:easy-schema or something else entirely.

does the schema you pass into a jam:method constructor have to use <Collection>.schema, or can you call it like this?

Currently, only a schema created with jam:easy-schema can be passed into schema using <Collection>.schema. One thing that might not be obvious is that jam:easy-schema smartly validates the data passed in so you don’t need to worry about passing in a subset of the schema. Just pass in the one schema you attach to the collection and it takes care of the rest.

My plan is to support other schemas. For example, I’ll likely add support for a zod schema. It’d be great to hear what other schemas people would like to have supported.

You bring up an interesting idea that I hadn’t considered before. I could enable what you describe but the schema object would need to be compatible with Meteor’s check. If you, or others reading this, like that idea, let me know. But at that point I wonder why not just use jam:easy-schema which does that for you with a bunch of other benefits and a nicer syntax.

Currently, if you want to use Meteor’s check, or any other validator function from a package of your choice, you can use the validate function instead of schema. Here’s an example with check:

import { createMethod } from 'meteor/jam:method'; // can import { Methods } from 'meteor/jam:method' instead and use Methods.create if you prefer
import { check } from 'meteor/check'

export const create = createMethod({
  name: 'todos.create',
  validate(args) { 
    check(args, {text: String}) 
  },
  run({ text }) {
    const todo = {
      text,
      done: false,
      createdAt: new Date(),
      authorId: Meteor.userId(), // can also use this.userId instead of Meteor.userId()
    }
    const todoId = Todos.insert(todo);
    return todoId;
  }
});

I like the idea of having a “standard” based on Meteor check for validation, separate from solutions like Zod, Typebox etc

. I like the simplicity of check which inspired jam:easy-schema.

it does some processing each time you pass in a “raw” schema

Not exactly. The processing happens once when you use attachSchema and then you retrieve it with <Collection>.schema.

I wonder if it’s worth exposing shapeSchema somehow to let people “compile” the schemas e.g. as a module export.

Sounds interesting. How would you imagine using this? What would be the benefits?

(I have more thoughts regarding easy-schema than jam:method at the moment, apologies!)

All good! Thanks again for sharing your thoughts.

[ceigey]

I wonder if it’s worth exposing shapeSchema somehow to let people “compile” the schemas e.g. as a module export.

Sounds interesting. How would you imagine using this? What would be the benefits?

Hmm, I think I misunderstood a bit about how the schemas were prepared from being a “POJO” to something that easy-schema actually uses (which I assumed shapeSchema was doing). I’m not sure how much overhead that introduces each time so I figure you probably only want to do it once since validation is quite expensive each time.

I think perhaps my ideal is something like this:

E.g.:

// don't mind the arbitrary data modelling choices...
let TodoSchema = {
  text: String,
  isCompleted: Boolean,
  completedAt: Date,
  createdAt: Date,
}

export const create = createMethod({
  name: 'todos.create',
  schema: TodoSchema, // No intermediate steps required
  // ...
}

// Then somewhere else
import { check } from 'meteor/check' // or 'meteor/jam:easy-schema';

check(someRandomData, TodoSchema);
// someRandomData is now seen as type { text: string; ... }

This has the fewest things for people to learn, and in a project I noticed a junior creating their own custom schemas in this way (using plain Meteor Check). It’s much cleaner looking than Zod, Typebox, Yup etc (albeit less powerful).

Assuming shapeSchema has some sort of a cost, and is used to prepare the POJO to another format which can be used by createMethod etc, then probably you’d have to do it this way if you wanted to avoid redoing work each time:

// don't mind the arbitrary data modelling choices...
import { createSchema } from  'meteor/jam:easy-schema';
let TodoSchema = createSchema({
  text: String,
  isCompleted: Boolean,
  completedAt: Date,
  createdAt: Date,
})

export const create = createMethod({
  name: 'todos.create',
  schema: TodoSchema, // Intermediate step performed AOT by createSchema
  // ...
}

// Then somewhere else
import { check } from  'meteor/jam:easy-schema'; // 'meteor/check' might not work I guess now

check(someRandomData, TodoSchema);
// someRandomData is now seen as type { text: string; ... }

I wanted to have a familiar behavior for people who choose to switch. I suppose I’d be surprised if there would be a collision — seems like you’d either use jam:easy-schema or something else entirely.

I think this is a tricky one in my head, because people might want to gradually migrate between the two. But I have to admit I’ve been lazy and can’t remember simple-schema’s API and how much overlap it has… maybe a migration is as simple as “simplifying” (heh) your simple-schema schemas (heh x2) and then quietly migrate behind the scenes just by changing packages?

Again, perhaps I’m not the target audience because I’ve been avoiding this by using Typebox.

[jam]

@ceigey

I think I misunderstood a bit about how the schemas were prepared from being a “POJO” to something that easy-schema actually uses (which I assumed shapeSchema was doing). I’m not sure how much overhead that introduces each time so I figure you probably only want to do it once since validation is quite expensive each time

shapeSchema does perform a conversion but it only happens once – when you attach the schema to the collection, not at the time of validation. Its cost is negligible.

Regarding your ideal, maybe a more detailed example would be helpful to illustrate. I think jam:easy-schema is closer to your ideal.

Your example:

Note that currently, jam:method doesn’t support passing in a schema like this, but it could.

import { Match } from 'meteor/check';
import { Todos } from './collection';
import { createMethod } from 'meteor/jam:method';
import { pick } from 'somewhere';

const TodoSchema = {
  _id: Match.Maybe(String),
  text: String,
  isCompleted: Boolean,
  completedAt: Date,
  createdAt: Date,
  note: Match.Maybe(String) // let's say you want to add an optional note to each todo
}

// passing in TodoSchema here would mean you'll need to pass in all the properties of TodoSchema as defined above into your method which probably won't be the case. 
// You could use a pick() function to pick the properties you'll pass in.
// Alternatively you could define a "mini" schema for each method based on what you pass in but that kind of defeats the point of having one TodoSchema.
export const create = createMethod({
  name: 'todos.create',
  schema: pick(TodoSchema, ['text']), // Meteor's check would be used automatically when the method is invoked. This doesn't happen currently but I could update jam:method to support it.
  run({ text }) {
    // ... //
  }
}

export const setCompleted = createMethod({
  name: 'todos.setCompleted',
  schema: pick(TodoSchema, ['_id', 'isCompleted']),
  run({ _id, isCompleted }) {
    // ... //
  }
}

With jam:easy-schema:

import { Todos } from './collection';
import { Optional } from 'meteor/jam:easy-schema';
import { createMethod } from 'meteor/jam:method';

// you'd most likely put the schema in it's own file or where you define the collection but putting here for illustration purposes.
const schema = {
  _id: String,
  text: String,
  isCompleted: Boolean,
  completedAt: Date,
  createdAt: Date,
  note: Optional(String) // let's say you want to add an optional note to each todo
}

// you'd have attachSchema where you define the collection but I have it here for illustration purposes
Todos.attachSchema(schema); // shapeSchema does its conversion as part of attachSchema

// all Todos methods will validate against the one Todos.schema
// jam:easy-schema automatically picks the properties that you pass in
export const create = createMethod({
  name: 'todos.create',
  schema: Todos.schema, // jam:easy-schema's check will be used automatically when the Method is invoked
  run({ text }) {
    // ... //
  }
}

export const setCompleted = createMethod({
  name: 'todos.setCompleted',
  schema: Todos.schema,
  run({ _id, isCompleted }) {
    // ... //
  }
}

Also note that jam:easy-schema:

1. Automatically validates on the server for write operations (for inserts / upserts data will be validated against the full schema). This is configurable so you can turn it off if you like.
2. Automatically generates a Mongo-compatible JSON Schema and attaches it to your db collection. This is configurable so you can turn it off if you like.

[marklynch]

Great work here @jam. We’re keeping tabs and debating if it might replace our existing validated methods. The before hook indeed would fit nicely with how we handle permissions. Just a few thoughts/questions.

I think the only package that uses attachSchema is aldeed:collection2 . If you switch to jam:easy-schema , you won’t need aldeed:collection2 because it has that functionality built in.

I had a look and didn’t see any mention of autovalues ? Other than the verification on insert/update that’s probably the most vital feature of collection2 for us. I’m sort of against them (for similar reasons I don’t like collection hooks) but for the basic createdAt, updatedAt, creatorId, updaterId it’s really useful.

I see your schema definition is basically tailored to working with check. It’s cool that we then get the param validation for free essentially as long as the params are included in the schema. I also really like that you’ve started taking advantage of the native mongo validation and can convert to jsonschema easily. I’m wondering if you’ve thought about form libraries? Those of us in Blaze land are tied to Autoform which is heavily tied to simpleSchema. Would it be easy to create a SS compatible schema to keep those library happy or easier to modify AF to work with ES ?

Also, in terms of being able to adopt a package like this a large part of it would be migrating existing schemas. You linked to some code from the latest version of SS which converts SS to JsonSchema. Do you think it would be easy to take that and modify it to create EasySchemas?
An alternate path would be if ES accepted it’s own format and also JsonSchema - that would be really cool and then I could see a relatively (famous last words ) simple migration path.

[jam]

Thanks Mark!

Quick note for others reading this thread: you can use jam:method without using jam:easy-schema. So if you’re happy with SimpleSchema or another validation package, you can use it with jam:method. We’re kind of discussing both in this thread, which I think is fine, but others might get confused. If you don’t like co-mingling the convo, there is a jam:easy-schema forum thread.

@marklynch

The before hook indeed would fit nicely with how we handle permissions.

Great to hear!

@marklynch

didn’t see any mention of autovalues

I don’t have plans to support autovalues in jam:easy-schema at this time. Like you said, I’m sort of against them. Having said that, can you give an example of how you use them and why you find them useful?

@marklynch

I’m wondering if you’ve thought about form libraries?

I haven’t really. My initial thought: I think these are so tied to the view layer you’re using that I think it’d be better to have someone write a package for their view layer that takes an Easy Schema as an input to magically generate a form.

@marklynch

a large part of it would be migrating existing schemas

In general, I think migrating from SimpleSchema to Easy Schema should hopefully be relatively straightforward. It could also make your schemas a lot more straightforward to grok. Though it will depend on how many features of SimpleSchema you rely on. Easy Schema intentionally doesn’t have all the SimpleSchema features.

@marklynch

You linked to some code from the latest version of SS which converts SS to JsonSchema

Hmm, not sure I’m following . Which code are you referring to?

@marklynch

An alternate path would be if ES accepted it’s own format and also JsonSchema

Easy Schema does a conversion to Mongo JSON Schema automatically so hopefully your use case is covered. Can you give an example of what you’re looking for?

[jam]

I made an update to support more schema packages.

The schema property in jam:method now supports:

• jam:easy-schema
• check
• zod
• simpl-schema

Here’s a quick example of each:

// jam:easy-schema
const create = createMethod({
  name: 'todos.create',
  schema: Todos.schema,
  run({ text }) {
    // ... //
  }
});

// check
const create = createMethod({
  name: 'todos.create',
  schema: {text: String},
  run({ text }) {
    // ... //
  }
});

// zod
const create = createMethod({
  name: 'todos.create',
  schema: z.object({text: z.string()}),
  run({ text }) {
    // ... //
  }
});

// simpl-schema
const create = createMethod({
  name: 'todos.create',
  schema: new SimpleSchema({text: String}),
  run({ text }) {
    // ... //
  }
});

If you’re using a different schema package, fear not, you can use validate to pass in a custom validation function instead:

// import your schema from somewhere 
// import your validator function from the package of your choice

const create = createMethod({
  name: 'todos.create',
  validate(args) {
    validator(args, schema), 
  },
  run({ text }) {
    // ... //
  }
});

If you have ideas for other improvements you’d like to see in jam:method, let me know!

[harry97]

@jam This is really awesome feature Great work brother!

@jkuester is there a way to create “bridges” uniforms style to create forms in autoform?

[guncebektas]

when I write meteor add jam:method, I get this error Error: Found multiple packages matching name: jam:method. Do you know anything else about this issue @jam

[jam]

Hmm that’s weird. I just tried to reproduce it but couldn’t.

I have a theory though. It’s kind of a long story but essentially I had unfortunate timing when I originally published the package – it happened to be during the Atmosphere outage on Oct 31. The Meteor team helped me resolve the issue and I republished the package which seemed to work correctly. Since you’re seeing this issue, I wonder if there is still something wonky.

@philippeoliveira do you have any ideas?

One thing you could do is remove Meteor’s local package db and reinstall Meteor (assuming you’re on Mac):

1. sudo rm /usr/local/bin/meteor
2. rm -rf ~/.meteor
3. curl https://install.meteor.com/ | sh

[guncebektas]

I have downloaded the source code under /packages folder and quickly implemented it with Zod. Your package seems SUPER.

[jam]

Awesome, thanks! Let me know if you have any ideas to improve it.

[janmp]

Just tried to install the package and got this:

13:10:32 in FamZ24 on  main [?] via ⬢ v16.16.0 using ☁️  default/ki-bot-395608 via 🅒 base took 2s 
➜ meteor add jam:method
/Users/jan-michaelpilgenroeder/.meteor/packages/meteor-tool/.2.14.0.9et9d9.ajq4o++os.osx.arm64+web.browser+web.browser.legacy+web.cordova/mt-os.osx.arm64/dev_bundle/lib/node_modules/meteor-promise/promise_server.js:218
      throw error;
      ^

Error: Found multiple packages matching name: jam:method
    at RemoteCatalog.getPackage (/tools/packaging/catalog/catalog-remote.js:602:13)
    at LayeredCatalog._returnFirst (/tools/packaging/catalog/catalog.js:190:32)
    at LayeredCatalog.getPackage (/tools/packaging/catalog/catalog.js:194:17)
    at /tools/cli/commands-packages.js:2207:59
    at Object.enterJob (/tools/utils/buildmessage.js:388:12)
    at /tools/cli/commands-packages.js:2197:20
    at Function.each (/Users/jan-michaelpilgenroeder/.meteor/packages/meteor-tool/.2.14.0.9et9d9.ajq4o++os.osx.arm64+web.browser+web.browser.legacy+web.cordova/mt-os.osx.arm64/dev_bundle/lib/node_modules/underscore/underscore-node-f-pre.js:1316:7)
    at /tools/cli/commands-packages.js:2196:7
    at Object.capture (/tools/utils/buildmessage.js:283:5)
    at Command.func (/tools/cli/commands-packages.js:2195:31)
    at /tools/cli/main.js:1534:15

[jam]

Thanks for reporting. I think this can only be fixed by the Meteor team. I opened a new issue to ask for their help.

In the meantime, you can use the package locally by copying it from GitHub and adding it to a packages folder at the root of your project.

Alternatively, I believe if you clear Meteor’s cache on your machine and reinstall Meteor, it should fix it:

@jam

One thing you could do is remove Meteor’s local package db and reinstall Meteor (assuming you’re on Mac):

1. sudo rm /usr/local/bin/meteor
2. rm -rf ~/.meteor
3. curl https://install.meteor.com/ | sh

[philippeoliveira]

Hello,

Could you try running the commands Jam mentioned and let us know if everything goes smoothly afterwards?

Has the Meteor installation on your MacOS been recent or has it been there for some time?

Best Regards,

Philippe Oliveira

[janmp]

There’s was no /usr/local/bin/meteor
Step 2 and 3 did the trick, though.

[jam]

I just published an update that allows for validating without executing the method.

Here’s a quick look:

// define a method
export const create = createMethod({
  name: 'todos.create',
  schema: Todos.schema,
  async run({ text }) {
    // ... //
  }
});

// validate against the schema without executing the method
create.validate({...})

// validate against only the relevant part of the schema based on the data passed in without executing the method
create.validate.only({...})

I’ll demonstrate soon how I plan to use .validate.only in particular for form UX.

[jam]

I’ve just published an update to make jam:method compatible with Meteor 3.0-rc.4, the latest release candidate.

If you like writing your methods isomorphically and are looking for a way to make your app feel instant for users, this section of the readme might be of interest to you.

[alimgafar]

Hey @jam

Would you be interested in doing a tutorial using your packages on the TWIM podcast?

Lmk.

[jam]

Hey Alim, it’d be fun to collaborate with y’all. Will send you a DM to talk more about it.