Zodern:relay - Typesafe Meteor methods and publications

zodern:relay is a new package for type safe Meteor methods and publications. It allows you to easily write schemas for methods and publications, and typescript will:

  • provide types for the method and publication parameters
  • validate the types of the arguments when calling the method or publication
  • provide the correct type for the method result

To use it for methods, you create your methods in files in methods directories:

import { createMethod } from 'meteor/zodern:relay';
import { z } from 'zod';

export const add = createMethod({
  name: 'add',
  schema: z.number().array().length(2),
  run([a, b]) {
    return a + b;
  },
});

Then, on the client you import the method and call it:

import { add } from '/imports/methods/add';

add([1, 2]).then(result => {
 console.log(result); // logs 3
});

Typescript provides the types for the run function’s parameter based on the schema, makes sure the types when calling add are correct, and uses the return type of the run function as the type for the method’s result.

It works very similarily for publications. You use createPublication in files in publications directories, and export a function to subscribe to the publication:

import { createPublication } from 'meteor/zodern:relay';
import { z } from 'zod';
import { Projects } from '/collections';

export const subscribeProject = createPublication({
  name: 'project',
  schema: z.object({
    id: z.string()
  }),
  run({ id }) {
    return Projects.find({ _id: id });
  },
});

Files in the methods and publications directories are treated specially. On the server they are left as is. For the client, their content is completely replaced so you don’t have to worry about exposing server code. Instead, on the client they export small functions to call any methods or subscribe to any publications that were originally exported.

More details are in the readme.

Credit

This uses a similar approach to @blitz/rpc. I found it on Monday, and thought something similar could work for Meteor.

16 Likes

Can you share what you would say are the key differences/advantages/use cases compared to simpl-schema w/validated method?

2 Likes

The advantage over validated methods is type safety.

Within the run function all of your parameters will have inferred types which the TypeScript type checker knows about and that you don’t have to specify manually. Also from what I understand from the README, the return type of the client side function will also be properly inferred to be the type that is returned on the server. If Meteor’s mongo implementation could somehow be updated so that the returns were properly typed to the shape of the returned object (the way Prisma does), then returned from a publication or method, that would mean that we could have type information inferred all the way from the database to the client without any manual type creation. This is kind of type inference is fantastic when working with typescript and I’ve only ever been able to have this end to end type inference when working with tRPC combined with Prisma.

2 Likes

Awesome! Going to implement it this week and try it out.

Awesome work as always @zodern :fire::fire::clap:t2::clap:t2:

Validate-methods also have type safety through the schema parameter

Yes, both packages do input parameter data validation. Validated method though doesn’t really provide what would be called type safety because the typescript compiler cannot infer type information from the validation. This package uses Zod to provide the data validation from which the typescript compiler can infer what types the input parameters will be and use that information to type check the rest of your code in the body of the run function. The other thing that this package does additionally, is it allows the type checker to infer the return type of the method across environmental boundaries. This means that if you return a Number from your run function, when you call that method on the client typescript will know the return of that function.

All of this can of course be specified manually. You could specify the types of the parameters, and you can cast the type for return value of the method, but thats laborious and you have to update it manually anytime there are changes to these types. That kind of developer experience is second to none and as the old saying goes “A luxury, once enjoyed, becomes a necessity”, you’ll not want to give it up once you’ve experienced it.

6 Likes

I think this is a good candidate to updating validated method.

4 Likes

Not only typescript can provide type safety to schemas but I believe I get what you mean.

We’re heavily invested in validated method. Seems a shame to replicate some similar functionality here. Is there a way to update VM with the typescript goodness using this approach ?

3 Likes

Thanks @copleykj. Your explanation is better than what I would have written.

Using zod for the schema is probably doable with validated method. It might even be possible as a mixin, depending on how the types for validated method are written.

One main difference between zodern:relay and validated methods is zodern:relay allows the types to work across environment boundaries, as @copleykj had mentioned.

To have the argument and return types when calling a method created with validated method, you have to import the method on the client to use the call function created for it. This both makes the method be a stub on the client, and includes the file’s content in the client. For some methods that might be fine, but for others it is not.

zodern:relay is the opposite - the method is only defined on the server, and is never in the client bundle. It is designed to make it impossible to accidentally include method or publication code on the client. You can only call createMethod and createPublication in methods or publications directories and have to use the babel plugin. Otherwise, your client will crash with an error explaining what happened and how to fix it.

Even though the code calling methods or subscribing to publications is on the client, and the definitions for the methods and publications are only on the server, the types still work. That is one of the main reasons I created this package, and the idea I borrowed from @blitz/rpc.

I have been considering ways to support method stubs. It needs to be clear when and exactly what code will be kept on the client. Two of my ideas are:

  1. Have a separate file extension to signify when the methods inside should also be used as method stubs on the client. For example the methods in projects.stub.ts would be defined on the server, and on the client as method stubs, while methods in projects.ts would only be defined on the server.
  2. The second idea is when creating a method, you can set a stub option to either a function to use as the stub on the client, or to true to use the run function as the stub on the client. The function used as the stub must be self contained - it can not use any variables or functions defined in the root of the file, and can not use any imports from files in methods or server directories.The only code kept on the client would be the stub function, and any imports it uses.
3 Likes

I had a bit more of a look through everything. This zod library is pretty cool.

Yeah, we prefer this way of working. We achieved this with VM by separating the method declaration and the run implementation. So our method can be used on the client but it only does type validation and permission checking and will only import and run the run code on the server. I can see why people might want stubs for some methods but would definitely argue that no-stub should be the default.

I guess the more fundamental difference between VM & relay then is the underlying schema library used, and zod is essentially what enables the cool functionality. We’re so tied to simple-schema that I don’t see any easy way to start using this.

A while ago I have seen someone (@radekmie was that you?) demonstrate how to get TS typings out of a data schema (I think that even was simpl-schema, wich is used in msg:validated-method). No Idea how that worked though and if it is viable.

This is great! Would be nice to take it a step further into the direction of this PR: Advanced meteor methods by Floriferous · Pull Request #11039 · meteor/meteor · GitHub

For example: adding hooks in methods is incredibly useful to create scalable codebases.

2 Likes

Yes, it was me and here it is: TypeScript 4.3 and SimpleSchema. It’s still possible and actually quite easy to do, even without a package. But I agree that packages like zod or typebox are better suited for that.

3 Likes

I’ve been considering various ways to handle the use cases of hooks and mixins while having type safety, and I’ve started working on pipelines: Pipelines by zodern · Pull Request #1 · zodern/meteor-relay · GitHub

Pipelines allow you to use an array of functions for a method or publication instead of a single run function. The output of each function is used as the input for the next function. The pipelines are composable - you can create functions or partial pipelines and use them for many methods and publications, and you can configure global pipelines that run before every method and publication.

For publications, some steps of the pipeline can become reactive and re-run when the results of queries change, allowing reactive joins and other uses.

The docs are complete, though some of the code hasn’t been written. If anyone wants to learn more, the docs are here. I am interested in hearing any opinions or suggestions on this. There are a number of benefits to pipelines, but there are also some parts that aren’t as nice.

6 Likes

I released version 1.1.0 today which adds the remaining features.

The name for methods and publications is now optional. It seemed unnecessary to name both the export, and the method/publication itself. Now the babel plugin will add a name if there isn’t one.

Method stubs are now supported. You define them in the same place as the method, and during build time the method stub and any imports used by the method stub will be extracted and used on the client.

import { Tasks, Events } from '/collections';

export const createTask = createMethod({
  schema: z.string(),
  stub: true,
  run(name) {
    Tasks.insert({ name, owner: Meteor.userId() });
  }
});

In this example, since stub is set to true the run function will be used as the method stub. Only the body of the run function and the Tasks import will be used on the client. All other code in the file will only exist on the server. Alternatively, stub can be a function if you don’t want to re-use the run function as the stub.

This version also adds pipelines, which allows you to break methods and publications into multiple steps. When used for publications, parts of the pipeline can be reactive, allowing you to change what cursors are published as other data in the database changes. Also, you can configure a global pipeline that runs before or after every method or publication.

8 Likes

Just started to implement this and migrate my validated methods over and I came here to say this is absolutely brilliant! It’s a joy to use!

@zodern
Do you have Github Sponsors or Patreon setup? If not, I can just subscribe to MontiAPM, but thought to ask.

1 Like

Jan, given that we just discussed taking simpl-schema package into the community (if someone takes it over and maintains it), is this a good idea with @zodern package now out and being maintained by him?

Secondly, what is the overall view of using meteor/check vs Zodern:relay?

I’m using meteor-check with some project specific extensions (meaning checks that are only important for my project).

I haven’t checked the Meteor documentation lately if that’s still somewhere being recommended or replaced by another package.

1 Like

Hi, I’m trying to use it inside a meteor package. However, I am getting the following error. I have a .babelrc file in my Meteor main directory and the required npm package is installed. I would be glad if you help.

Error: Method name is missing. Do you have the relay babel plugin configured?
    at createMethod (packages/zodern:relay/server-main.ts:60:11)
    at module (packages/bordo:ticket-system/lib/methods/tickets/create.ts:13:22)
    at fileEvaluate (packages/modules-runtime.js:336:7)
    at Module.require (packages/modules-runtime.js:238:14)
    at require (packages/modules-runtime.js:258:21)
    at /Users/recepozen/bordo/whatsgoo-main/.meteor/local/build/programs/server/packages/bordo_ticket-system.js:369:1
    at /Users/recepozen/bordo/whatsgoo-main/.meteor/local/build/programs/server/packages/bordo_ticket-system.js:377:3
    at /Users/recepozen/bordo/whatsgoo-main/.meteor/local/build/programs/server/boot.js:369:38
    at Array.forEach (<anonymous>)
    at /Users/recepozen/bordo/whatsgoo-main/.meteor/local/build/programs/server/boot.js:210:21
    at /Users/recepozen/bordo/whatsgoo-main/.meteor/local/build/programs/server/boot.js:423:7
    at Function.run (/Users/recepozen/bordo/whatsgoo-main/.meteor/local/build/programs/server/profile.js:256:14)
    at /Users/recepozen/bordo/whatsgoo-main/.meteor/local/build/programs/server/boot.js:422:13