Looking for help migrating packages to Meteor 3.0

3.0 migration
41

I never wanted to comment on this, but for what it’s worth, here is my opinion: schema checks, whether in Meteor/MongoDB or any other system, should not involve async calls to HTTP endpoints or to the database. Just think about a schema check on an array of items, what if you have a custom function hitting the database for each item? In 12 years working with Meteor it has not occurred to me that one could/should be going for this kind of approach. Gee, Simple Schema, while great, is enough of a slow hog as it is.

I personally see it as a great ergonomic adage, made possible back in the day because of the synchronous isomorphic code / Fibers everywhere / “so why not” kind of thinking. Which, I assume, led many to use this pattern. But maybe it’s one of those things that should now be left behind.

While I can see why and where it would be useful, it looks like an antipattern and maybe it should be highlighted as such; if these async calls are indeed required, maybe they actually belong to a different layer (authorisation, for instance). But that is just me, and likely I don’t have the full perspective on why this is a good idea.

Anyway, a more useful question is: would some extra sponsorship/funding help move things forward, whatever the decision regarding Simple Schema?

2 Likes
42

Deciding to move a “validation” to another part of your code because it is async or not is not really about being good pattern or anti pattern or good thinking - in simple terms, it is called a tool/library/framework limitation.

[Addendum]

43

I should have made it clear that my point was in reference to using Simple Schema in conjunction with Collection2. In this case, I stand by what I said: it is an antipattern to make async calls while schema checking a document about to be inserted or modified in said MongoDB. If you can point me to a database system allowing this pattern as standard, I will stand corrected.

The rest of use cases, where Simple Schema is used for validating method arguments and other such use cases, yes, I have no comment there, though personally I avoid that too.

Nothing changes the fact that Collection2 is now being held back unnecessarily.

1 Like
44
45

Not sure if this was intended as a reply to my post. If so, I asked for an example of “a database system”. Collection2 + SimplSchema was (and still is) meant to fulfil the role of schema enforcement in the context of database operations. Allowing async calls in this context is simply handing over a big footgun to users, some of whom won’t have a second thought about race conditions and other such troublesome concepts.

46

Again, these are tools/libraries. It depends on developers how they will use or mis-use a tool. I can use it differently on how you can use it.

How it works is simple as indicated in the example of an async validator posted above. But I will repeat the example here:

E.g.
When inserting a new document to Collection A, it requires a referenceId from Collection B. Therefore, it is required that the referenceId must exist before inserting.

There are multiple ways to do this just like:

  1. Adding a check everytime an insert/update is done
  2. Add it in a shared insert/update function
  3. Add it in the schema validation

Since you are looking for something built-in, you can do the example above in Oracle DB using a foreign key constraint: constraint

47

Don’t get me wrong, I understand all the benefits you have listed. But…

Oh, there is no shortage in the Meteor ecosystem of such examples, with insecure and autopublish being just a couple that come to mind. The result is that the internet is full of complaints of Meteor being insecure :man_shrugging:

I am more than familiar with the concept.

My point was that, since async custom validator functions in SimplSchema make it difficult to update Collection2, maybe it’s a good idea to at least park the attempt for now. Your suggestion in the other thread, to have a preliminary release, is spot on.

48

Here’s our current use case and let me know how you’d handle it:

We use a (reactive) AutoForm form where a user can be assigned to an organization based on a certain criteria.

allowedValues() {
            // we don't want to show ourselves and no other satellite pharmacies.
            const query = { isSatelliteOf: { $exists: false } }
            if (Meteor.isClient && getCurrentlyEditedInstitutionId())
               query._id = { $ne: getCurrentlyEditedInstitutionId() }

            return _.map(
               Collections.Institutions.find(query, {
                  sort: { name: 1 },
                  fields: { _id: true }
               }).fetch(),
               "_id"
            )
         },
         autoform: {
            options() {
               // we don't want to show ourselves and no other satellite pharmacies.
               const query = { isSatelliteOf: { $exists: false } }
               if (Meteor.isClient && getCurrentlyEditedInstitutionId())
                  query._id = { $ne: getCurrentlyEditedInstitutionId() }

               return _.map(
                  Collections.Institutions.find(query, {
                     sort: { name: 1 },
                     fields: { _id: 1, name: 1 }
                  }).fetch(),
                  (institution) => {
                     return {
                        label: institution.name,
                        value: institution._id
                     }
                  }
               )
            }
         }```
49

I will be honest - we never used this kind of approach other than exclusively on the client.

At illustreets, we created a wrapper around SimplSchema that allows returning static schemas, or even building them on the fly and passing any arguments that may need to be dynamic, at run time (and yeah, many times from some async call - we had hundreds of those even before the Fibers crisis). Example:

import { IlluSchemas } from 'meteor/illustreets:schemas';

// a static schema
IlluSchemas.addSchema('CreateVisArgs', {
  sourceSlug: {
    label: 'Data source',
    type: String,
  },
  filter: {
    type: String,
    optional: true,
    autoform: {
      type: 'hidden',
      label: false,
    },
  },
});
// then somewhere else...
IlluSchemas.validate('CreateVisArgs', someObject);

// creates schema on-the-fly
export function getColumnsSchema(columns, prop, aliases) {
  const fields = {};
  columns.forEach((col) => {
    fields[col.name] = {
      type: Boolean,
      label: aliases[col.name],
      optional: true,
      autoform: {
        id: `${col.name}_${prop}`,
        defaultValue() {
          return col[prop];
        },
      },
    };
  });
  return IlluSchemas.getSchema(fields);
}
// then elsewhere, on client...
const columns = Collection.findOne({ ... })?.columns;
// ... prop, aliases
getColumnsSchema(columns, prop, aliases).validate(someObject);

// on server...
const columns = (await Collection.findOne({ ... }))?.columns;
// ... prop, aliases
getColumnsSchema(columns, prop, aliases).validate(someObject);

This approach gives us the flexibility to still cache and reuse fields and entire schemas, and a few other features.

I don’t mind mixing up minimongo with SimplSchema on the client, given that minimongo is also a client construct, an in-memory structure instantiated just for the current user, at the end of the day. Also, I allow the client to be messy, cause most frontend stuff ends up messy somehow. But the server is a different beast. MongoDB is shared among multiple users, processes, and workers.

Also, the client part of your code really does not even need the Meteor.isClient check. The autoform block only runs on client, unless you are really trying something unholy. At illustreets we have multiple schemas with autoform blocks that are being populated just like that, used in common server/client code. But if we needed a shared allowedValues field with values from Mongo, we would generate the schema on the fly via IlluSchemas with allowedValues containing just an array of values passed as argument.

Oh, and here is a very recent comment from the creator of SimplSchema about the perils of trying to make allowedValues (and I assume other functions) async: AutoValue using async functions · Issue #484 · longshotlabs/simpl-schema · GitHub. Nearly impossible. It will simply mess up client side validations.

[EDIT]
I should add, obviously dynamic schemas like I describe above won’t work with Collection2, but for the cases where we need something like in your example, we run a validation in the services layer, just before inserting/updating the document. Collection2 would use a static version of the schema, to ensure the document has the expected structure.

50

These pull requests have now all been merged. Many thanks to @storyteller and @denyhs!
CC @harry97 @sabativi @filipenevola

Correction: the changes for tmeasday:publish-counts have been approved, but the PR is not yet merged.

2 Likes
51

Super glad you followed up on the request. Great work :clap: :clap:

You’re one of the very people who actually chipped in :sweat_smile:

2 Likes
52

This one also merged recently thanks to @grubba & @denyhs

2 Likes
53

@storyteller and @jkuester - I know you are both busy these days, but I just noticed your discussion from last year under Archive? · Issue #13 · Meteor-Community-Packages/meteor-stylus · GitHub. If you still want to archive it, before you do it, please merge the two pull requests and publish the new package versions.

1 Like
54

Would be good to have such included in a single reference file (like the Excel sheet that Harry started), especially with local forks as otherwise people will waste time reinventing the wheel (meaning updating the package to version 3 compatibility).

Is that everyone else’s views as well?

1 Like
55

If there’s one area where it would bring immense value having someone from the core meteor team be assigned the responsibility, this is it. Project management just does not work well when it relies on people’s free time and competing with the more “fun” parts, like writing code to make packages 3.0-compatible.

Keeping the Google Sheet alive and up-to-date, proactively investigating forks and new versions, reaching out to people and asking them to republish when needed, providing advise and having some master migration documentation with links.

Please?

3 Likes
56

Well, this has been a point of critique of Meteor and Tiny for a long time.

I guess some things never change.

57

I’ve been busy upgrading our application to the latest RC but I thought it’s worth mentioning that activitree/scss doesn’t support meteor packages import @import '{meteoric124:meteoric-sass}/scss/ionic'; It’s still a good package and you should migrate to it if you’re not using file imports.

But if you still do, like me, I’ve started a new PR that updates it to work the latest RC, explaining what needs to be done to get it working, thanks!

1 Like
58

A small word to confirm the pwix-blaze-layout PR has been merged, and the new v2.3.2 published.
Thanks to Manuel.
(unfortunately, github didn’t notify me when the PR has been submitted, so, sorry for the delay)
Best regards
Pierre

2 Likes
59

I can recommend leonardoventurini:scss instead of waiting for a working Meteor 3-compatible release of fourseven:scss.

It works with meteor 3 and doesn’t have the nasty activitree:scss showstopper bug

2 Likes
60

@leonardoventurini if you have a bit of time it would be great if you could compare what you have with what @harry97 is working on:

so that we can get this done asap.