Also, something that @alimgafar mentioned was that it might be better to run the schema validation on the app server before hitting DB (as that costs data transfer and then processing power on the database). Which is what collection2 is doing today.
1 Like
And at that point I’d like to point again to a solution using mixins so those who want to use Mongo’s built-in shema can use it or others who want to use something else on the server can use this.
1 Like
It would be great if you can still keep SimplSchema as an option. The main advantage of it is that one can have a schema attached to a collection, then reuse it in one or more forms (reactively), and even use pick or omit to reuse parts of it for validation in specific methods. It is unbeatable in that respect.
1 Like
As a follow-up to this discussion:
@jkuester have made SimpleSchema Meteor 3.0 “compatible” but async computations are not supported. That might be an intermediate step for Collection2 i.e. Meteor 3.0 compatible without async computations.
Then, the remaining features will wait for the full-async version of SimpleSchema.
P.S.
I am not familiar with other features of Collection2 outside the simple automated schema validation (which must become async to support async computations).
1 Like
“Bring your own validation” looks like the least requirement for maintenance with the widest compatibility to existing validation tools. At the very least, a before hook with error handling will suffice.
5 Likes
This has not yet been brought up for discussion internally.
I will say my thoughts currently on every point.
Integration into Meteor core.
Does not sound good because of the arguments brought up and also because the whole 3.0 migration focus on bringing Meteor back to the full javascript ecosystem. A package being in the core does not mean that you cannot use other package that solves your problem but for me gives the signal that the package that is in the core, is the default way to solve this problem. In this case there are plenty of options I think, in our ecosystem and in the broader JS ecossytem.
- Integrate/Merge with Astronomy as mentioned here
This one I like the most, when I first started with meteor I was looking for something like Mongoose but with a focus on meteor(today I prefer just using the driver but I can see those who like using an ODM).
- Same as Astronomy but Mongoose
This one is nice. It supports change streams but I would like to see how would feel to use the real-time features for example, with astronomy works how we would in Meteor normally. Does that occur with the Mongoose package?
Maintain the status quo and evolve independently.
Maybe this is the path where is more natural? Removing the usage of simple schema sounds like the next step in the package history. Bring your own validation sounds like the today default way of having a schema-like library, many js package does this, you bring/pass the validator and the package just uses the function to validate.
1 Like
Yes, we first used Collection2 + Simple Schema but never really liked it. Astronomy was a very good alternative for it, especially if you like the whole model concept. We are currently using our own model layer with Joi validation because Astronomy isn’t maintained anymore.
I would prefer Mongoose as replacement because it is widely used within the Javascript community, and if you have some micro services or other apps which share the same database like your Meteor app, it would be very nice if you could use your Mongoose model there too.
1 Like
I never used Astronomy, but as for Collection2 (or universe:collection; it’s the same idea) and Mongoose – the performance impact is huge. I removed them (or recommended to) from a lot of projects (Collection2 from Meteor ones and Mongoose from non-Meteor ones). The biggest cost is the copying after fetch (transform in Meteor and its counterpart in Mongoose).
With that in mind, I’d say I’m not against those in Core (I don’t mind), but they should be opt-in, or at least opt-out.
4 Likes
I wanted to give this thread some time so everybody can join, so thanks a lot to everybody for sharing their opinion.
- Core integration
I guess given the fact this wasn’t raised internally by the core team as @grubba mentioned and how this gets in the way of other options, then it’s safe to rule out, at least for now.
- Mongoose Integration
This’d have to be ruled out too, given how this’s already implemented thanks to @paulishca, no need to repeat the efforts.
I believe Manuel’s comment sums up the case for Collection2 pretty nicely:
it’s a relic of a bygone era. It was meant to make up for lack of schema enforcement in MongoDB.
So that leaves us with other two which I’m yet undecided about, but I can assure you all who use Collection2 that the project is still maintained and compatible with Meteor 3.0.
TBH, I’ve never used Astronomy and the issue of resource allocation that @storyteller mentioned is still a thing. We’ll see.
EDIT: What’re the Meteor specific features that Astronomy offers than no other ODM library offers? @diavrank95
Please if anyone knows more about this point, share your knowledge with us.
If those features are so good to pass on then we might consider maintaining/migrating it to 3.0
2 Likes
the performance impact is huge . I removed them (or recommended to) from a lot of projects (Collection2 from Meteor ones and Mongoose from non-Meteor ones). The biggest cost is the copying after fetch (
transformin Meteor and its counterpart in Mongoose)
Performance is a key metric for sure but not all projects are under that much of a heavy load that they’d have to squeeze out every single ounce of CPU so for some it’s ok to use an ODM.
though I guess you agree with us on not doubling down on a single option especially when it comes to the core.
With that in mind, I’d say I’m not against those in Core (I don’t mind), but they should be opt-in, or at least opt-out.
1 Like
I like the idea of adding a simple before hook to Meteor core that anyone can leverage. Maybe it should be called validate to differentiate it from other hooks. This keeps things simple and adds the most flexibility. I think the trick would be to ensure that it would run before any other collection-hooks.
I would definitely take advantage of that hook for jam:easy-schema which like Collection2 validates automatically before DB writes.
Agreed, there is definite benefit here not to mention the ability to customize errors much more easily. Mongo’s JSON Schema errors are pretty lackluster imo.
Agree. An ODM is not my cup of tea but to each their own and I think it’s great if there’s a go to package for those that like it.
————
Not to take the convo away from Collection2 but I wanted to highlight a couple things for people who are evaluating which direction to go when it comes to validation:
jam:easy-schema generates a MongoDB JSON Schema automatically based on a schema with a check-like syntax that you attach to your Collection.
jam:easy-schema also has this ability. In fact, if you use it in conjunction with jam:method the pick-ing is done for you automagically. 
If you haven’t checked it out yet, it’s worth a look to see if it’s a good fit for you. If you already evaluated it and decided it’s not for you, let me know. I may be able to incorporate what you need.
I think the main thing that jam:easy-schema doesn’t have at this point that was mentioned above is an Autoform integration. I don’t think it would be difficult though. I’m using Svelte as my view layer though and I have a different approach that I’ll be sharing with the community soon.
2 Likes
I may have been unnecessarily harsh 
. It is a relic, if one strictly needs just for MongoDB schema validation. But in tandem with SimplSchema, it gives you a lot of centralised control over validation. You can reuse the same schema, or parts of it, for: client forms, validating method arguments, validating the full document at the point of insertion/update.
I don’t think you have a better combo covering all these at the same time.
And I think it should be like that for as long as SimplSchema is relevant for forms.
Now you made me curious . How would it work with autoform? If not, does it work with any forms package? I looked at it when you announced it, but haven’t seen any mentions in the repo.
EDIT: I don’t know how I managed to overlook the “Autoform integration” part of your post.
1 Like
After a little bit of reading Astronomy docs The best thing Astronomy has is the holistic approach to managing collections related code. To achieve a similar experience using Collection2, you’d have to pair it with Simple Schema, collection-helpers, collection-hooks, collection-instances and at least a package for methods validations like validated methods or jam:method.
There has been a previous attempt by Quave to circumvent this issue by offering one package that leverages these packages underneath to offer a more streamlined DX.
Given what I had just mentioned, I think the next logical step would be to bring all these libraries together:
into one single mono-repo/single package where by installing Collection2, you get the benefits of all three libraries. We won’t bundle simple-schema as we’ll be allowing you to bring your own validation whether it be easy-schema, zod or anything else. The nice thing about this is backwards comparability too!
I believe this is the natural next step as it offers better DX not just for you, the end user of Collection2, but also us the maintainers (@storyteller, @jkuester and me). During the 3.0 migrations @jkuester suffered a lot when he was trying to update collection-instance due to its cross-reliance on other packages. Also, this serves the point @storyteller mentioned about lessening the amount of packages we’re responsible for. And, finally we’d end up with something similar to Astronomy yet fitting for 3.0 that utilizes our collective efforts we’ve put so far.
wdyt?
2 Likes
I’m using simpl-schema and collection2 heavily. If this was not working anymore, it would be a showstopper for any Meteor 3 upgrade.
1 Like
It’s great to discuss about the Meteor community packages and ideas for its future.
In my experience using Collection2, I have felt it a heavy piece, tightly linked with conventions like mandatory adoption of simple schema. I’ve faced challenges on the past with unsynchronized method interfaces compared to core Meteor expectations or performance impact, although it’s been a while since I used it, making it hard to provide specific examples. Likely the package has improved a lot. I also know that it has been crucial for many companies to adopt this package and its conventions, allowing them to focus on their products, which is the core concept of the Meteor framework.
Considering your proposed directions, I suggest keeping Collection2 as a separate entity maintained by the community. If it becomes more compact and loosely connected, embracing ideas like detaching simple schema and opting for custom validators, it would capture my interest definitely.
I agree with your view that integrating this package and the proposed changes into the core would demand significant effort from a small core team. Since there are enough challenges on the core ahead already, such as modern bundlers, native experience enhancements, and performance, which surely are more important for eveyone using Meteor.
3 Likes
Thank you for chiming in, Nacho.
Considering your proposed directions, I suggest keeping Collection2 as a separate entity maintained by the community.
So, this means you oppose merging it with other packages, correct?
If it becomes more compact and loosely connected, embracing ideas like detaching simple schema and opting for custom validators, it would capture my interest definitely.
Decoupling it from simple-schema seems like the way to go forward yet I wanted to do this step only after figuring out what to do with the package.
1 Like
Do you mean merging Collection2 into the core? I believe that conversation has been deferred for any package, given the current focus on Meteor 3.x.
In my opinion, not the final decision or representing the core team’s stance, we should consider migrating packages that are sufficiently decoupled, lightweight, critical, widely used, and preferably easy to maintain in the short term. This could help alliviate the core team’s workload as they concentrate on other priorities such as bundler, native, performance enhancements, among others.
Simultaneously, the community can lead the evolution of concepts in different packages. At some point, considering moving them into the core could be considered, but it’s not always necessary. Recommendations can be made in the official docs, and maintenance can still be handled by the community or individuals.
Edit: Or by merging you refer to what you stated here, Collection2 Future - #19 by harry97
I think the direction can be to merge them as well. I am not too familiar with all these packages anymore, some I never use them, and others I found alternatives. But if merging them serves well to the users of these packages, and it simplifies setup and experience, I would say go for it. The concept can evolve and even simplified with merging them. And with a simple sight I can see these packages to act and expand over the same concepts of managing collections and data. Makes sense.
1 Like
I would love the option to use Collection 2 with Zod!
At the moment I’m duplicated all SimplSchema schemas with zod and that opens up potential for errors (as well as duplicating work).
Just to clarify things as many people here already mentioned big issues if C2 would get breaking changes. The perspectives of Meteor Community Packages is compatibility-first and @storyteller and I try to make things as non-breaking as possible unless we are forced to.
Currently the bigger issue is async computations in simple schema that would also have consequences to Collection 2 as the whole validation for Simple Schema would be async.
A simple example: a field is optional if user has a role x. A field with such computed optional property may then look like this:
isAllowed: {
type: Boolean,
optional: async () => {
return !Roles.userIsInRole(Meteor.userId(), 'canDoThin', 'admins')
}
}
* Please don’t pin me on the usecase for the above code