New package - jam:easy-schema - An easy way to add schema validation

[jam]

jam:easy-schema is an easy way to add schema validation. It’s lightweight and fast.

I was looking for an easy way to define a schema in one place, use it for validation on Meteor Methods, and use it to create a Mongo JSON Schema for the db. Rather than cobble together a bunch of packages, some of which are heavy and seem outdated, I built this one.

It also automatically validates against the schema on the server before hitting the db to provide better error messaging since the Mongo errors can be a little lacking.

It could serve as a replacement for SimplSchema and Collection2, depending on how you use those packages.

It’s highly configurable so you can customize to your liking.

If you end up taking it for a spin, let me know! If you have ideas on how to make it better, feel free to post below, send me a DM, or open up a Github discussion.

[guncebektas]

Well done the package is very impressive

[tcastelli]

Looking really good! Have you thought about adding typescript support or how would that work?

[harry97]

I gotta be honest with you, I really like the schema package a lot more than the methods package. The fact that it generates a JSON schema and the check-like interface makes it a pleasure to work with. Good job

[copleykj]

No Typescript support?

[jam]

I’ve thought about it. I think for the Typescript aficionados I’d rely on zod to create the schema. There’s one missing piece to the puzzle here though which is a nice zodToMongoJsonSchema function. I’m hoping that this package will fill in that gap. Then I would likely release a separate package maybe called easy-schema-zod. Let me know if you have any thoughts here.

Other than that, if something like this proposal is ever introduced, it’d be trivial to add types to the package as is I think.

[marklynch]

Continuing from the jam:method thread as it’s true all my questions are schema related!

I don’t have plans to support autovalues in jam:easy-schema at this time. Like you said, I’m sort of against them. Having said that, can you give an example of how you use them and why you find them useful?

We have a base schema that we build other schemas upon which basically handles a few common variables. Here’s what it looks like. There are a handful of other places but as a general rule we try avoid ‘magic’ and things happening outside the main execution path.

import SimpleSchema from "simpl-schema";
SimpleSchema.extendOptions(["autoform"]);
export const BaseSchema = new SimpleSchema({
    createdAt: {
        type: Date,
        optional: true,
        autoValue: function() {
            if (this.isInsert) {
                return new Date();
            } else if (this.isUpsert) {
                return {$setOnInsert: new Date()};
            } else {
                //allow createdAt date to be set by migrations
                //this.unset();
            }
        }
    },
    updatedAt: {
        type: Date,
        optional: true,
        autoValue: function() {
            //both updating and inserting should modifying the updated date.
            if (this.isUpdate || this.isInsert || this.isUpsert) {
                return new Date();
            }
        }
    },
    creatorId: {
        type: String,
        regEx: SimpleSchema.RegEx.Id,
        optional: true,
        autoValue: function() {
            if (this.isInsert) {
                return this.userId;
            } else if (this.isUpsert) {
                return {$setOnInsert: this.userId};
            } else {
                this.unset();
            }
        }
    },
    updaterId: {
        type: String,
        regEx: SimpleSchema.RegEx.Id,
        optional: true,
        autoValue: function() {
            if (this.isInsert || this.isUpdate || this.isUpsert) {
                return this.userId;
            }
        }
    },
    deleted: {
        type: Boolean,
        optional: true
    }
});

I haven’t really. My initial thought: I think these are so tied to the view layer you’re using that I think it’d be better to have someone write a package for their view layer that takes an Easy Schema as an input to magically generate a form.

Yes, but as you’re generating the jsonSchema that might be an easier path as a lot of form libs support that.

In general, I think migrating from SimpleSchema to Easy Schema should hopefully be relatively straightforward. It could also make your schemas a lot more straightforward to grok. Though it will depend on how many features of SimpleSchema you rely on. Easy Schema intentionally doesn’t have all the SimpleSchema features.

It would be cool maybe to have a comparison table in the repo? To jog the memory of long time users who have maybe forgotten what magic they are using from SS. I’m just thinking about the custom validation function now and that could be a problem. Do you have any solution to interdependent fields? So if type = x then status is required or that type of rule?

Hmm, not sure I’m following . Which code are you referring to?

This code here

Easy Schema does a conversion to Mongo JSON Schema automatically so hopefully your use case is covered. Can you give an example of what you’re looking for?

As it looks like it might be easy to migrate SS → JsonSchema I was just wondering if having the ability to define EasySchemas in JsonSchema might make life easier. But yeah, that loses the brevity of easyschema and is probably quite a bit of work (particularly infering the check side of things from the jsonschema) so probably makes no sense for this package.

[jam]

@marklynch

as a general rule we try avoid ‘magic’ and things happening outside the main execution path

Cool, thanks for sharing that example. I could see how that would be handy. This type of functionality kinda feels more in the realm of collection hooks. My understanding is that the collection hooks functionality will be brought into Meteor core in the not too distant future. When it is, I believe jam:easy-schema will be able to validate automatically before those write operations without needing to change anything but I’ll be sure to test it when Meteor releases collection hooks.

jam:easy-schema is in line with your philosophy of trying to avoid too much “magic”.

@marklynch

as you’re generating the jsonSchema that might be an easier path as a lot of form libs support that

Interesting. I haven’t spent any time looking into that but maybe I should. Thanks for the suggestion. If you have specific ideas on how this could work, let me know.

@marklynch

It would be cool maybe to have a comparison table in the repo? To jog the memory of long time users who have maybe forgotten what magic they are using from SS

Good idea. I might add something like this if there’s enough interest from people currently using SimpleSchema. Generally, jam:easy-schema seeks to be compatible with Mongo JSON Schema and tries to avoid a lot of the “magic” that SimpleSchema has which could be handy but also led to a heavier, slower package (there’s always tradeoffs). Having said that, I’m always open to ideas on ways to improve the package and if there is critical functionality missing, let me know.

@marklynch

Do you have any solution to interdependent fields?

Not at the moment. If this is crucial, let me know and I can take a closer look at what it would take to add it. I believe this would be equivalent to using dependencies in Mongo JSON Schema.

@marklynch

I was just wondering if having the ability to define EasySchemas in JsonSchema might make life easier.

Ah ok, I see what you’re saying. Funnily enough, I actually started creating this package using JSON Schema as the starting point. As I dug into it more, I realized that I didn’t really love its verbosity. I liked starting with the brevity of a check-like syntax that I could then convert to Mongo JSON Schema so that I could have the best of both worlds. It also enables this type of scenario:

Let’s say you’re building something new but you’re not sure what shape your data may take

1. Start prototyping with check to get something in a customer’s hands
2. Once things have solidified a bit, easily move to jam:easy-schema for better data integrity

[generalledger]

One of the things I love about zodern:relay is that it provides the return types of subscriptions and methods in client side code. And, as long as I do my part, the types pass through the pipeline too. This is an astonishing improvement in DX.

From what I’ve been reading in this thread, your package looks really great, but not having types is a major deal breaker for me, personally.

Regardless, thank you for creating this. I’ll be watching!

[jam]

Thanks for sharing. I’ll likely take a closer look at adding typescript support. I noticed that the check package does support type narrowing so hopefully I can piggyback off of that. If you, or someone else reading this who’s a typescript expert, are interested in lending a hand, I’d welcome it.

[jam]

I just added a new feature: where – a custom validation function which includes support for interdependent fields. Here’s a quick example of making status required when there’s text:

{
  // ... //
  text: Optional(String),
  status: {type: Optional(String), where: ({text, status}) => {
    if (text && !status) throw EasySchema.REQUIRED // you can throw a custom message here if you'd like
  }},
  // ... //
}

Thanks to @marklynch for suggesting the feature and providing additional help!

[jam]

A couple of recent updates

1. jam:easy-schema is now Meteor 3.0 compatible. It’s featured in @fredmaiaarantes’s Simple Tasks which recently migrated to 3.0.

2. You can now customize error messages and it comes with even nicer default error messages.

// this will use the default error messages
const schema = {
  email: {type: String, min: 1, regex: /@/}
}

// customizing error messages is as easy as doing this
const schema = {
  email: {type: String, min: [1, 'You must enter an email'], regex: [/@/, 'You must enter a valid email']}
}

[jam]

A few recent updates to jam:easy-schema . It now has:

1. Better typescript and code completion support
2. An optimized jam:method integration
3. A changelog

It also has a tiny footprint if you choose to use it client-side. It’s roughly 1/10th the bundle size compared to alternatives.

[jam]

I just released a new update to jam:easy-schema with two new features:

1. Throwing all errors for a better UX. If your user is submitting a form and it has multiple errors, you can display all errors to them rather than forcing them to go one by one through the errors, resubmitting the form each time (ugh). Maybe you’re thinking: the package didn’t already do this? In earlier versions of this package, this feature simply wasn’t possible because Meteor’s check didn’t support it. Thankfully my PR to check to enable it was merged into Meteor 2.16
2. Support for Mongo.ObjectID, which of course includes support for it with Mongo’s JSON Schema. Here’s a quick look on how to use it:

import { ObjectID } from 'meteor/jam:easy-schema';

const schema = {
  _id: ObjectID,
  // ... rest of your schema //
}

[jam]

I just released a big update to jam:easy-schema with the following features:

1. Fluent syntax to set conditions (optional). Here’s a quick example:

import { has } from 'meteor/jam:easy-schema';

const schema = {
  text: String[has].min(1).max(140) // string with at least 1 character and a max of 140 characters
}

2. Set default values

Defaults can be static or dynamic. Static defaults will be set once, when the doc is inserted. Dynamic defaults will be set on each write to a doc. Pass in a function to make a default dynamic.

import { has, ID } from 'meteor/jam:easy-schema';

const schema = {
  creatorId: ID[has].default(Meteor.userId), // static
  updaterId: ID[has].default(() => Meteor.userId()), // dynamic
  username: String[has].default(Meteor.user), // static, value will be Meteor.user.username
  done: Boolean[has].default(false), // static
  createdAt: Date[has].default(Date.now), // static
  updatedAt: Date[has].default(() => Date.now()) // dynamic
}

3. Easily configure a base schema

import { EasySchema } from 'meteor/jam:easy-schema';

const base = {
  createdAt: Date,
  updatedAt: Date
}

EasySchema.configure({ base }); // all schemas will inherit this base

4. New ID type for Meteor-generated _ids

import { ID } from 'meteor/jam:easy-schema';

const schema = {
  _id: ID, 
  // ... //
}