Looking for help migrating packages to Meteor 3.0

[illustreets]

I should have made it clear that my point was in reference to using Simple Schema in conjunction with Collection2. In this case, I stand by what I said: it is an antipattern to make async calls while schema checking a document about to be inserted or modified in said MongoDB. If you can point me to a database system allowing this pattern as standard, I will stand corrected.

The rest of use cases, where Simple Schema is used for validating method arguments and other such use cases, yes, I have no comment there, though personally I avoid that too.

Nothing changes the fact that Collection2 is now being held back unnecessarily.

[rjdavid]

Ajv JSON schema validator

[illustreets]

Not sure if this was intended as a reply to my post. If so, I asked for an example of “a database system”. Collection2 + SimplSchema was (and still is) meant to fulfil the role of schema enforcement in the context of database operations. Allowing async calls in this context is simply handing over a big footgun to users, some of whom won’t have a second thought about race conditions and other such troublesome concepts.

[rjdavid]

Again, these are tools/libraries. It depends on developers how they will use or mis-use a tool. I can use it differently on how you can use it.

How it works is simple as indicated in the example of an async validator posted above. But I will repeat the example here:

E.g.
When inserting a new document to Collection A, it requires a referenceId from Collection B. Therefore, it is required that the referenceId must exist before inserting.

There are multiple ways to do this just like:

1. Adding a check everytime an insert/update is done
2. Add it in a shared insert/update function
3. Add it in the schema validation

Since you are looking for something built-in, you can do the example above in Oracle DB using a foreign key constraint: constraint

[illustreets]

Don’t get me wrong, I understand all the benefits you have listed. But…

@rjdavid

It depends on developers how they will use or mis-use a tool

Oh, there is no shortage in the Meteor ecosystem of such examples, with insecure and autopublish being just a couple that come to mind. The result is that the internet is full of complaints of Meteor being insecure

@rjdavid

Since you are looking for something built-in, you can do the example above in Oracle DB using a foreign key constraint: constraint

I am more than familiar with the concept.

My point was that, since async custom validator functions in SimplSchema make it difficult to update Collection2, maybe it’s a good idea to at least park the attempt for now. Your suggestion in the other thread, to have a preliminary release, is spot on.

[harry97]

Here’s our current use case and let me know how you’d handle it:

We use a (reactive) AutoForm form where a user can be assigned to an organization based on a certain criteria.

allowedValues() {
            // we don't want to show ourselves and no other satellite pharmacies.
            const query = { isSatelliteOf: { $exists: false } }
            if (Meteor.isClient && getCurrentlyEditedInstitutionId())
               query._id = { $ne: getCurrentlyEditedInstitutionId() }

            return _.map(
               Collections.Institutions.find(query, {
                  sort: { name: 1 },
                  fields: { _id: true }
               }).fetch(),
               "_id"
            )
         },
         autoform: {
            options() {
               // we don't want to show ourselves and no other satellite pharmacies.
               const query = { isSatelliteOf: { $exists: false } }
               if (Meteor.isClient && getCurrentlyEditedInstitutionId())
                  query._id = { $ne: getCurrentlyEditedInstitutionId() }

               return _.map(
                  Collections.Institutions.find(query, {
                     sort: { name: 1 },
                     fields: { _id: 1, name: 1 }
                  }).fetch(),
                  (institution) => {
                     return {
                        label: institution.name,
                        value: institution._id
                     }
                  }
               )
            }
         }```

[illustreets]

@harry97

allowedValues() {
  // we don't want to show ourselves and no other satellite pharmacies.
  const query = { isSatelliteOf: { $exists: false } }
  if (Meteor.isClient && getCurrentlyEditedInstitutionId())
    query._id = { $ne: getCurrentlyEditedInstitutionId() }

  return _.map(
    Collections.Institutions.find(query, {
      sort: { name: 1 },
      fields: { _id: true }
    }).fetch(),
    "_id"
  )
}

I will be honest - we never used this kind of approach other than exclusively on the client.

At illustreets, we created a wrapper around SimplSchema that allows returning static schemas, or even building them on the fly and passing any arguments that may need to be dynamic, at run time (and yeah, many times from some async call - we had hundreds of those even before the Fibers crisis). Example:

import { IlluSchemas } from 'meteor/illustreets:schemas';

// a static schema
IlluSchemas.addSchema('CreateVisArgs', {
  sourceSlug: {
    label: 'Data source',
    type: String,
  },
  filter: {
    type: String,
    optional: true,
    autoform: {
      type: 'hidden',
      label: false,
    },
  },
});
// then somewhere else...
IlluSchemas.validate('CreateVisArgs', someObject);

// creates schema on-the-fly
export function getColumnsSchema(columns, prop, aliases) {
  const fields = {};
  columns.forEach((col) => {
    fields[col.name] = {
      type: Boolean,
      label: aliases[col.name],
      optional: true,
      autoform: {
        id: `${col.name}_${prop}`,
        defaultValue() {
          return col[prop];
        },
      },
    };
  });
  return IlluSchemas.getSchema(fields);
}
// then elsewhere, on client...
const columns = Collection.findOne({ ... })?.columns;
// ... prop, aliases
getColumnsSchema(columns, prop, aliases).validate(someObject);

// on server...
const columns = (await Collection.findOne({ ... }))?.columns;
// ... prop, aliases
getColumnsSchema(columns, prop, aliases).validate(someObject);

This approach gives us the flexibility to still cache and reuse fields and entire schemas, and a few other features.

I don’t mind mixing up minimongo with SimplSchema on the client, given that minimongo is also a client construct, an in-memory structure instantiated just for the current user, at the end of the day. Also, I allow the client to be messy, cause most frontend stuff ends up messy somehow. But the server is a different beast. MongoDB is shared among multiple users, processes, and workers.

Also, the client part of your code really does not even need the Meteor.isClient check. The autoform block only runs on client, unless you are really trying something unholy. At illustreets we have multiple schemas with autoform blocks that are being populated just like that, used in common server/client code. But if we needed a shared allowedValues field with values from Mongo, we would generate the schema on the fly via IlluSchemas with allowedValues containing just an array of values passed as argument.

Oh, and here is a very recent comment from the creator of SimplSchema about the perils of trying to make allowedValues (and I assume other functions) async: AutoValue using async functions · Issue #484 · longshotlabs/simpl-schema · GitHub. Nearly impossible. It will simply mess up client side validations.

[EDIT]
I should add, obviously dynamic schemas like I describe above won’t work with Collection2, but for the cases where we need something like in your example, we run a validation in the services layer, just before inserting/updating the document. Collection2 would use a static version of the schema, to ensure the document has the expected structure.

[illustreets]

@illustreets

reywood:publish-composite # Awaiting review from MCP maintainers
tmeasday:publish-counts # Awaiting review from MCP maintainers
useraccounts:core # Pull request sent to Meteor Compat Packages
useraccounts:bootstrap # Pull request sent to Meteor Compat Packages

@illustreets

json-routes@v3.0.0 # Pull request submitted to Meteor Compat Packages
rest-json-error-handler@v1.1.3 # Pull request submitted to Meteor Compat Packages

These pull requests have now all been merged. Many thanks to @storyteller and @denyhs!
CC @harry97 @sabativi @filipenevola

Correction: the changes for tmeasday:publish-counts have been approved, but the PR is not yet merged.

[harry97]

Super glad you followed up on the request. Great work

You’re one of the very people who actually chipped in

[illustreets]

@illustreets

Correction: the changes for tmeasday:publish-counts have been approved, but the PR is not yet merged.

This one also merged recently thanks to @grubba & @denyhs

[illustreets]

@illustreets

Update 2:

coagmano:stylus@v1.1.3 # Pull request submitted to MCP
coagmano:stylus@v2.0.3 # Pull request submitted to MCP

@storyteller and @jkuester - I know you are both busy these days, but I just noticed your discussion from last year under Archive? · Issue #13 · Meteor-Community-Packages/meteor-stylus · GitHub. If you still want to archive it, before you do it, please merge the two pull requests and publish the new package versions.

[a4xrbj1]

@illustreets

We forked locally and migrated a bunch of packages:

Would be good to have such included in a single reference file (like the Excel sheet that Harry started), especially with local forks as otherwise people will waste time reinventing the wheel (meaning updating the package to version 3 compatibility).

Is that everyone else’s views as well?

[permb]

If there’s one area where it would bring immense value having someone from the core meteor team be assigned the responsibility, this is it. Project management just does not work well when it relies on people’s free time and competing with the more “fun” parts, like writing code to make packages 3.0-compatible.

Keeping the Google Sheet alive and up-to-date, proactively investigating forks and new versions, reaching out to people and asking them to republish when needed, providing advise and having some master migration documentation with links.

Please?

[a4xrbj1]

@permb

If there’s one area where it would bring immense value having someone from the core meteor team be assigned the responsibility, this is it. Project management just does not work well when it relies on people’s free time and competing with the more “fun” parts, like writing code to make packages 3.0-compatible.

Well, this has been a point of critique of Meteor and Tiny for a long time.

I guess some things never change.

[harry97]

I’ve been busy upgrading our application to the latest RC but I thought it’s worth mentioning that activitree/scss doesn’t support meteor packages import @import '{meteoric124:meteoric-sass}/scss/ionic'; It’s still a good package and you should migrate to it if you’re not using file imports.

But if you still do, like me, I’ve started a new PR that updates it to work the latest RC, explaining what needs to be done to get it working, thanks!

[pwix]

A small word to confirm the pwix-blaze-layout PR has been merged, and the new v2.3.2 published.
Thanks to Manuel.
(unfortunately, github didn’t notify me when the PR has been submitted, so, sorry for the delay)
Best regards
Pierre

[permb]

I can recommend leonardoventurini:scss instead of waiting for a working Meteor 3-compatible release of fourseven:scss.

It works with meteor 3 and doesn’t have the nasty activitree:scss showstopper bug

[storyteller]

@leonardoventurini if you have a bit of time it would be great if you could compare what you have with what @harry97 is working on:

Make it compatible with 3.0

so that we can get this done asap.

[harry97]

Keep in mind, current meteor-scss still uses node-sass - npm while Leo’s and activitree fork use Sass: Dart Sass which is an entirely different library.

[leonardoventurini]

@storyteller @harry97 I left some comments in the PR. But that’s mainly it, I am using a JavaScript-only implementation of SASS, and it’s also using it’s sync API which is also faster. The tests work fine in Meteor 3 and the package imports also seem to work fine in my fork. Hope it helps.