Meteor Roadmap Update, August 27th, 2024

@leonardoventurini

Is the plan to bring all Auth to Meteor core?

RocketChat has many auth packages.

WeKan uses older version of ldapjs npm package. Ldapjs is not maintained anymore.

WeKan has fork of OAuth2/OIDC at wekan-accounts-oidc and wekan-oidc directories for Auth0, ADFS 4.0, Azure AD B2C, Google, RocketChat, GitLab, NextCloud and Zitadel. There is also package for Sandstorm login.

List of various login methods is at right menu topic “Login Auth” at wiki:

https://github.com/wekan/wekan/wekan/wiki

Also there is plans to add SAML and CAS, but any existing code is still Meteor 2 only I think. Question is, how to port to Meteor 3.

If Meteor would have most login methods at core, then it could be similar like Ruby on Rails that has OmniAuth packages for various auth methods:

Replacing the embedded babel+reify that is part of isobuild with something else might be good in the long run but “here be dragons”.

reify generates wrappers and extra code for every code file in a meteor application and enables some additional features:

  • nested imports (a proposed addition to javascript that never went anywhere)
  • dynamic imports
  • top-level await
  • async-tagged code blocks that work well with fibers (meteor < 3)

If you’ve never looked into the generated code, here’s an example of what reify does (meteor 3.0.1).
Note all the module calls which are part of the reify/meteor runtime: module.link, module.wrapAsync, etc

original code:

import axios from "axios";

const getAndrei = async () => {
  const response = await axios.get('https://api.github.com/users/alisnic')
  return response.data
};

const Andrei = await getAndrei();

export default Andrei;

Generated code inside app.js:

!module.wrapAsync(async function (module, __reifyWaitForDeps__, __reify_async_result__) {
  "use strict";
  try {
    let axios;
    module.link("axios", {
      default(v) {
        axios = v;
      }
    }, 0);
    if (__reifyWaitForDeps__()) (await __reifyWaitForDeps__())();
    const getAndrei = async () => {
      const response = await axios.get('https://api.github.com/users/alisnic');
      return response.data;
    };
    const Andrei = await getAndrei();
    module.exportDefault(Andrei);
    __reify_async_result__();
  } catch (_reifyError) {
    return __reify_async_result__(_reifyError);
  }
  __reify_async_result__()
}, {
  self: this,
  async: false
});
2 Likes

All? No, that is unrealistic, not to mention a hell to maintain. You are reading too much into it. I can foresee strategic expansion like Apple OAuth. LDAP and SAML could make sense in the future.
There is a difference between core and community maintained auth packages. Core are should be things that almost everyone is going to use or will be required in certain contexts (like Google and Apple OAuth) and then infrastructure to build additional auth packages if you need them (OAuth core packages).

We want to provide everything the user needs to have a complete experience, while at the same time moving away from maintaining things we don’t need to maintain.

I also believe some centralization for such packages would make sure they are well maintained and documented.


Our tests show the same thing, also for methods. Reactive flows however, show some degradation, which ends up affecting reactivity dense/large scale apps.

Right now we recommend using SERVER_WEBSOCKET_COMPRESSION=false if your app fits that category, as it gives a lot of breathing room. We have opted to not release a beta fully removing it as some users might still benefit from compression.

The problem I believe is in the sheer amount of calls the mongo logic triggers, it can trigger many thousands of tasks and async resources in just a few seconds, combine that with ALS/AH, publish composite or polling, you can get an explosive combination.

Both Meteor 2 and Meteor 3 are prone to oplog flooding, however Meteor 3 seems be more easily affected, and also it is reported to show increased CPU in normal levels of reactivity.

In essence, the observer/mongo logic is a strong candidate for further optimization, but there might be some tradeoffs we need to make regarding bindEnvironment or whatever else relies on ALS.

It is very hard to “prove” that ALS is the culprit because we can’t easily turn on/off, we can experiment with it in one place or the other. In one experiment I did for removing it from AsynchronousQueue tasks I got a reduction of 20mb in RAM use, but no apparent CPU change, and it is a very simple use case. I assume those 20mb freed GC from running too. Multiply that by 10/20x…

This needs further research, and I would appreciate help with it.

4 Likes

How many Meteor users will need this?

Can you let us know how long your build time is? I don’t share that feeling and I don’t see that Meteor has become worse. But then we only have about 250-300k LOC.

Also didnt find Meteor slow, its much faster with Vite though.
I think build times are the last thing I care about as long as they’re “average across js”

1 Like

It might be unpopular opinion, but I dont think Meteor core should compete with basically everything JS-related on the internet. Awesome and predictable realtime performance out of the box and wonderful DX, - all meteor really needs, especially since the more popular Meteor is, the more solutions/boilerplates will community generate naturally.

The better Meteor is for realtime apps, with heavy and frequent data updates through subscriptions, the more sense it makes as unique solution.

I dont think we should focus on beginners and extensively cover bases like REST-API. Web development has much matured, there’s NestJS for overcomplicated JS backend(Meteor would work great as module for it btw) and tons of tools like Nextjs or Golang Gin server, which are incredibly easy to learn and maintain.

To my knowledge there arent many good out of the box solutions for writing realtime-oriented apps. Especially if you dont have a full fledged team of developers with frontend/backend/devops and message-queue expertise. You can absolutely lose your mind trying to do a simple chat module or voting system. There’re many new toolsets to bootstrap improvements in Meteor and arent yet adopted by big frameworks well, like aforementioned Nats/Jetstream KV or React 19 stuff like useOptimistic – React or even suspended rendering at this point. Though Blaze has niches where its unique and truly convinient, like Email templating, especially with AI assistance.

It could be best to grab exciting stuff ASAP and set things aBlaze

@Mousy

I dont think we should focus on beginners and extensively cover bases like REST-API

My Meteor 2 app has REST API. I did not remember where I did see how to do REST API with Meteor 3, so I asked Meteor 3 Docs AI, and it replied with this REST API code example:

import { LinksCollection } from "/imports/api/links";
import { WebApp } from "meteor/webapp";

WebApp.handlers.get("/all-links", async (req, res) => {
  const links = await LinksCollection.find().fetchAsync();
  res.json(links);
});

// other code ...
1 Like

It would be nice if Meteor’s WebApp provided ways of easy authentication/authorization users. In my dream world there would be some kind of a middleware available for WebApp that would consume request’s headers (auth headers) and enrich the request passed to the route handler with either user ID (so something similar to this context in Meteor methods) or authenticated user object from the Meteor.users collection.

@dweller23

The way to implement that is:

  1. Use Meteor Roles to define roles.
  2. Have Login API that can be used with username and password to get time limited Bearer Token, that is saved to database.
  3. For other API, check that it includes Bearer token, that bearer token has not expired, check Meteor Roles is API user allowed to do that API call, and do that API call.
  4. Maybe log API calls to MongoDB database or elsewhere.

I have something similar implemented for my Meteor 2 app, but it is using hardcoded permissions and old REST API package, it’s not so useful for Meteor 3, where is better to use Meteor Roles.

Curious, how would having this be useful for you? Do you have a specific scenario in mind?

I just had to implement something like this for interfacing with ApolloGraphQL, because the calls to Apollo come in via a WebApp listener. Here’s the code in case it may be helfpul.

SERVER SIDE

import {ApolloServer} from "@apollo/server";
import {expressMiddleware} from "@apollo/server/express4";
import {ApolloServerPluginDrainHttpServer} from "@apollo/server/plugin/drainHttpServer";
// import express from "express";
import http from "http";
import cors from "cors";
import {PubSub, withFilter} from "graphql-subscriptions-continued";
import {WebSocketServer} from "ws";
import {useServer} from "graphql-ws/lib/use/ws";
import {makeExecutableSchema} from "@graphql-tools/schema";
import {MongodbPubSub} from 'graphql-mongodb-subscriptions';
import {getUserIdByLoginToken} from "./utils_server/meteor-apollo-utils";
import typeDefs from "./api/schema";
import {resolvers} from "./api/resolvers";
import {array_of_cors_origins} from "../both";
const requestIp = require('request-ip');

const schema = makeExecutableSchema({typeDefs, resolvers});

// https://www.youtube.com/watch?v=AcZ5dcYMwA4
// for non-Meteor users, use this:
// const app = express();

// https://forums.meteor.com/t/working-apollo-client-server-setup-with-context-for-both-queries-and-subscriptions/61770/2?u=vikr00001
const meteorInternalVersionOfExpress = WebAppInternals.NpmModules.express.module;
const app = meteorInternalVersionOfExpress()
const httpServer = http.createServer(app);
const wsServer = new WebSocketServer({
    path: "/graphql",
    noServer: true,
});

//https://forums.meteor.com/t/apollo-server-express-server-setup-for-subscriptions/57871/2?u=vikr00001
WebApp.httpServer.on("upgrade", function upgrade(request, socket, head) {
    if (!request.url) {
        return;
    }
    switch (request.url) {
        case "/graphql":
            return wsServer.handleUpgrade(request, socket, head, function done(ws) {
                wsServer.emit("connection", ws, request);
            });
        default:
            break;
    }
});


const serverCleanUp =useServer({
    schema,
    context: async (ctx, msg, args) => {
        let context = await getDynamicContext(ctx, msg, args, true)
        return context;
    },
}, wsServer);

// https://forums.meteor.com/t/solved-check-ip-address-on-api-applications/55443
function parseIp(req) {
    let response =
    (typeof req?.headers['x-forwarded-for'] === 'string'
        && req?.headers['x-forwarded-for'].split(',').shift())
    || req?.connection?.remoteAddress
    || req?.socket?.remoteAddress
    || req?.connection?.socket?.remoteAddress;

    return response;
}

async function getDynamicContext(ctx, msg, args, isSubscription){
    let token = null;
    if (isSubscription){
        token = ctx.connectionParams.authorization
    }
    else{
        token = ctx.req.headers['token']
    }

    let user = null;
    let userId = null;
    try {
        if ((!!token) && (token !== "null")) {
            [user, userId] = await getUserIdByLoginToken(token);
        }
    } catch (error) {
        console.log('context: ', error)
    }

    let clientIp = '';
    try {
        let request = ctx?.req || ctx.extra.request;
        clientIp = request?.clientIp || parseIp(request) || request?.ip || request?.headers['x-forwarded-for'] || request?.connection?.remoteAddress  || ctx?.res?.req?.headers["x-forwarded-for"] || request?.client?.remoteAddress;
        console.log("in getDynamicContext. clientIp:", clientIp);
    } catch {
        console.log("in getDynamicContext. Couldn't get clientIp")
    }

    return {
        user: user,
        userId: userId,
        clientIp: clientIp
    };
}

const apolloServer = new ApolloServer({
    schema,
    plugins: [
        ApolloServerPluginDrainHttpServer({httpServer}),
        {
            async serverWillStart() {
                return {
                    async drainServer() {
                        await serverCleanUp.dispose();
                    }
                }
            }
        }
    ],
});

await apolloServer.start();

WebApp.handlers.use(
    "/graphql",
    cors({
        origin: function (origin, callback) {
            const allowedOrigins = array_of_cors_origins;
            if (!origin || allowedOrigins.indexOf(origin) !== -1) {
                callback(null, true);
            } else {

                console.log('origin: ', origin)
                console.log('array_of_cors_origins: ', array_of_cors_origins)
                callback(new Error('Not allowed by CORS'));
            }
        },
        credentials: true,
    }),
    meteorInternalVersionOfExpress.json(),
    requestIp.mw(),
    expressMiddleware(apolloServer, {
        context: async (ctx, msg, args) => {
            const context = await getDynamicContext(ctx, msg, args, false);
            return context;
        }
    })
);

CLIENT SIDE

import {ApolloClient, split, HttpLink, InMemoryCache, ApolloLink, ApolloProvider, createHttpLink} from "@apollo/client";
import {GraphQLWsLink} from "@apollo/client/link/subscriptions";
import {setContext} from "@apollo/client/link/context";
import {getMainDefinition} from "@apollo/client/utilities";
import {createClient} from "graphql-ws";
import {graphQLQueryEndpoint, graphQLSubscriptionEndpoint} from "../both";


const wsLink = graphQLSubscriptionEndpoint
    ? new GraphQLWsLink(
        createClient({
            url: graphQLSubscriptionEndpoint,
            connectionParams: () => {
                return {
                    authorization: localStorage.getItem("Meteor.loginToken") || "",
                };
            },
            on: {
                error: props => {
                    console.log("wsLink error", props);
                },
            },
        }),
    )
    : null;

const httpLink = createHttpLink({
    uri: graphQLQueryEndpoint,
    fetchOptions: {
        rejectUnauthorized: false,
    },
});


// Middleware to add custom headers
const customHeadersMiddleware = new ApolloLink((operation, forward) => {
    // Define your custom headers
    const customHeaders = {
        "token": localStorage.getItem("Meteor.loginToken")
    };

    // Use operation.setContext to add the custom headers to the request
    operation.setContext(({ headers }) => ({
        headers: {
            ...headers,
            ...customHeaders,
        },
    }));

    return forward(operation);
});

const splitLink = split(
    ({ query }) => {
        const definition = getMainDefinition(query);
        // console.log('definition.kind: ', definition.kind)
        // console.log('definition.operation: ', definition?.operation)
        return (
            definition.kind === 'OperationDefinition' &&
            definition.operation === 'subscription'
        );
    },
    customHeadersMiddleware.concat(wsLink),
    customHeadersMiddleware.concat(httpLink)
)

const apolloClient = new ApolloClient({
    link: splitLink,
    cache: new InMemoryCache({}),
});


export {apolloClient};

You can see that on the client side, it grabs the token of the logged-in user, and adds it to the header, here:

    const customHeaders = {
        "token": localStorage.getItem("Meteor.loginToken")
    };

    // Use operation.setContext to add the custom headers to the request
    operation.setContext(({ headers }) => ({
        headers: {
            ...headers,
            ...customHeaders,
        },
    }));

Then on the server it gets the current Meteor user by a call to:

getUserIdByLoginToken()

Which comes from:

import {getUserIdByLoginToken} from "./utils_server/meteor-apollo-utils";

Getting the logged in user when using the share target api from a PWA is a scenario I have in my app. The users upload images via the share functionality on the phones and I need to know which users it belongs to. I am using a function I got from the forum as middleware in meteor 2. Having it available from the core (or a package) in meteor 3 would be a nice addition.

The ideal way of handling this through a PWA is by using a service worker to intercept the POST and let the logged-in session in the client upload the file through the client app.

1 Like

@vikr00001 that’s nice, thanks for sharing, does not seem as daunting as I imagined.

It would be nice if WebApp had such functionality out of the box, because there’s also a matter of typescript and proper types, which will bloat the shared code a bit probably.

1 Like

Are there plans of switching from Cordova to Capacitor?

Candidate items

  • Integrate with Tauri, it might replace Cordova and Electron in a single tool
  • Support building mobile apps using CapacitorJS

Could you please post or link to the function?

The function is based on this post: Authenticating a user in server-side routes - #5 by jamgold

const getUserIdFromToken = loginToken => {
  const user = Meteor.users.findOne(
    { 'services.resume.loginTokens.hashedToken': Accounts._hashLoginToken(loginToken) },
    { fields: { _id: 1 } }
  );

  if (user) {
    return { response: 'success', message: user._id };
  }
  return { response: 'error', message: 'noUser' };
};

The loginToken is stored in cookies by the serviceWorker:

import { Meteor } from 'meteor/meteor';
import { Accounts } from 'meteor/accounts-base';
import { Cookies } from 'meteor/ostrio:cookies';

Meteor.startup(() => {
  Accounts.onLogin(user => {
    const cookies = new Cookies();
    cookies.set('meteor_login_token', user.token);
  });
  navigator.serviceWorker
    .register('/sw.js', { scope: '/pwa' })
    .catch(error => {
      console.log('ServiceWorker registration failed: ', error);
    });
});

The share target is definedin in the manifest:

    "share_target": {
      "action": "./pwaShareChecklistImage",
      "method": "POST",
      "enctype": "multipart/form-data",
      "params": {
          "files": [{
              "name": "file",
              "accept": ["image/*"]
          }]
      }
    }

To get the userId from the WebApp endpoint:

WebApp.connectHandlers.use('/pwaShareChecklistImage', (req, res, next) => {
  const loginToken = req.cookies?.meteor_login_token;
  const { response, message } = getUserIdFromToken(loginToken);
}
1 Like