Migrating an Atmosphere package to NPM

If I would to migrate some of my Atmosphere package to NPM, what are current best practices to do this? Can community point me to some resources? I have few questions:

  • can a NPM package depend on a core Meteor package?
  • how you talk to Meteor Mongo?
  • create publish endpoints and Meteor methods?
  • how can I use/reuse Tinytest tests I made for my Atmosphere packages?
  • and what if my packages are written in CoffeeScript, how can I make NPM packages written in CoffeeScript? do I have to manually compile it in JavaScript before I release them?
5 Likes

With a little bit of shoehorning, yes, you can do that.

can a NPM package depend on a core Meteor package?

take a look these lines where the npm package depends on a meteor package

how you talk to Meteor Mongo?

I’d say, the same as above since Mongo is just a named export of a meteor package.

create publish endpoints and Meteor methods?

Again, Meteor is the main export of a core meteor package. The main challenge here would probably be getting the server and browser separation right. Apparently a long standing bug that prevented compatibility with npm’s solution is soon to be resolved.

how can I use/reuse Tinytest tests I made for my Atmosphere packages

good question, this would probably require you to adapt a unit testing library but I doubt it would be too hard to find one with similar syntax.

and what if my packages are written in CoffeeScript, how can I make NPM packages written in CoffeeScript

in your package.json you declare the entry point(s) (browser/node) of your app and that entry point should be valid javascript. that means you need to integrate a build tool into your workflow. But rest assured that this is standard workflow because even if you did not use coffescript, you would be transpiling from ESwhatever to ES5. This is the typical npm workflow. npm, by use of package.json, allows you to define pre-publish hooks to automate that for you.

6 Likes

So it looks like this is going to be repeated a lot

try {
    if (typeof Package === 'object') {
        if (Match === undefined) {
            Match = Package['check'];
            Match = Match.Match;
        }

        if (SimpleSchema === undefined) {
            SimpleSchema = Package['aldeed:simple-schema'];
            SimpleSchema = SimpleSchema.SimpleSchema;
        }
    }

If MDG is serious about the migration to npm it’d be nice if they (or someone else) put that logic into an npm package so you could do something like this

import { getMeteorPackage } from 'get-meteor-package';
const SimpleSchema = getMeteorPackage('aldeed:simlpe-schema');

food for thought

2 Likes

Hm, above is how one imports a Meteor package, not how one make a package depends on it, no? How can I assure that some core Meteor package is loaded and populates Package before my NPM package loads? Or that it is even added to client, for example. Imagine that I have an app which uses only my NPM module, and that module depends on some core Meteor package which is not included otherwise to client. How can I make it be included?

I really hope that, some time in the future, a tool is developed to assist with migrating Atmosphere packages to NPM.

It seems like it’s going to become a necessity in the future once full NPM support is released.

2 Likes

I’d like to add the question: how do I separate code for different target platforms, i.e. server vs. browser vs. cordova? For me, this is one of the core features of Meteor packages, especially if you’re developing for both mobile and web.

2 Likes

Afaik, there’s no way to actually depend on a meteor package from an npm package and have the dependency included in meteor’s dependency tree.

But the other way around is also “somewhat” true. In order to avoid having multiple copies of npm dependencies in your meteor app, you should generally not use npm depends but instead use the check-npm-versions package, which lets the developer know that they have some missing dependencies.

I belive this is an acceptable compromise.

Well, you can do it only for browser/node => client/server but unfortunately, there’s no cordova identifier. But I guess you may be able to use some other environment detection library for mimicing similar feature. Although, granted, it would now allow proper codebase separation as Meteor does for your.

Why don’t you do it yourself? You can even publish it on npm for much fame and appreciation :wink:

1 Like

NPM used to support peer dependencies for this problem but that support was dropped in npm 3.0, so this isn’t even possible with NPM at the moment since I believe you’re describing a peer dependency.

One not-so-perfect solution could be to check for Meteor package dependencies at runtime. You’d just add a method to your application that will throw a descriptive error when it’s evaluated if you’re missing a dependency. So remember how I described something like this to access Meteor packages?

import { getMeteorClientPackage,  getMeteorServerPackage } from 'meteor-package';
const SimpleSchema = getMeteorClientPackage('aldeed:simlpe-schema');
const cluster = getMeteorServerPackage('meteorhacks:cluster');

What if inside the getMeteorPackage implementation, it would check if the package exists via global.Package[name] in the appropriate environment (client or server), and if it’s not found throw an error to kill the app process and log some descriptive error like

$: ==> Meteor Package missing: 'aldeed:simple-schema' on client environment.
       Please run 'meteor add aldeed:simple-schema' to satisfy the Meteor 
       package dependency.

And the implementations look like this, to check in each environment

function reportMissingMeteorDependency(name, environmet) {
     throw new Error(`
==> Meteor Package missing: '${name}' on ${environment} environment.
       Please run 'meteor add${name}' to satisfy the Meteor 
       package dependency.
              `
           );
}

function getMeteorClientPackage(name) {
   if (Meteor.isClient) {
      try {
        return global.Package[name];
      } catch (e) {
          reportMissingMeteorDependency(name, 'client');
      }
   } 
}
function getMeteorServerPackage(name) {
   if (Meteor.isServer) {
      try {
        return global.Package[name];
      } catch (e) {
          reportMissingMeteorDependency(name, 'server');
      }
   } 
}

That way it would log which environment the package was missing in. This doesn’t work until runtime, but it would get the job done in a fairly graceful way.

5 Likes

Am following this thread with much interest. Would be very interested to hear @sashko’s and other MDG member opinions on this.

6 Likes

@mitar @serkandurusoy

here’s my first draft which implements what I described above https://www.npmjs.com/package/meteor-package

on the server when an error is thrown it just keeps trying to rerun over and over and logging the error, if there were a way to just force Meteor to kill the process altogether that’d be better

1 Like

No, I am thinking of a normal dependency. So by defining a normal dependency, that dependency should be loaded before your code, not after. This is how Meteor has been doing things.

Now, how to prevent the same (or very similar, but not exactly the same) dependency to be loaded multiple times is another problem. Especially because some Meteor packages (like Tracker) require to be loaded only once.

1 Like

@efrancis this is really cool! It looks super awesome!

Mind if I ask why you did not include window for the browser global so that this could also work - at least similarly, for runtime that is - with client-only packages?

Furthermore, I think, although might be tricky with load order or bundle optimization, you may define an “import proxy” kind of export that works like

import { Meteor, Mongo } from 'meteor-package/core-packages'
import { HTTP, Email } from 'meteor-package/core-addons'

and perhaps with some additional work and maintenance

import { SimpleSchema, Cluster } from 'meteor-package/community-addons'

and the latter could actually be automated using atmosphere’s ddp endpoint which one could access from a CI tool and a ddp client.

@mitar the solution from @efrancis would be there to report missing dependencies only, there’s currently no way to actually ensure/load missing dependencies.

I understand that about the solution from @efrancis. And this is also why probably there it seems to be a need for some core development before NPM packages could really replace Atmosphere packages.

1 Like

I guess it would not be too far off here to say that for that kind of proper progression, the long awaited complete move to npm (meteor core packages on npm) is necessary.

But I should admit that the more I use npm, the more I feel like atmopshere is a more developer-friendly package system :confused:

3 Likes

Yea, I like atmosphere packages as well. Especially when you consider their support for multiple architectures.

4 Likes

I don’t believe that’s necessary, part of Meteor trying to make client and server act the same is that on the client global === window, they’re aliases for each other. But if Meteor packages are ever available outside of a Meteor context that may no longer be true, so I’ve just published 0.1.1 that uses window on the client instead.

Yea, I like atmosphere packages as well.

Me too :\ it has some features that npm just can’t provide an alternative to yet. I’m hoping this proposal or something similar gains traction this year, but it doesn’t seem very prioritized.

2 Likes

Well, perhaps we’re not to wait too long at least from a meteor-support perspective: Preserve true "main" and "browser" fields of package.json modules. by benjamn · Pull Request #8213 · meteor/meteor · GitHub

1 Like

Yeah without internal changes to npm we’re stuck with a runtime solution instead of at dependency load time. Here are a couple ideas, neither of which I’m in love with:

  • I suppose someone could make a wrapper around npm that helps with resolving Atmosphere packages, akin to how yarn wraps npm
  • when the meteor-package npm package above sees a missing dependency missing and logs the error asking the user to run "meteor add <name>" it could theoretically spawn a child process and run the “meteor add” command for you, which would automate the process more, but that is admittedly really hacky and doesn’t solve different version requirements (luckily Meteor has done epic work to keep things backwards-compatible). I believe the meteor dev server process would need to be entirely killed and restarted as well
1 Like

Shoot, wish I had found this before I published my Galaxy maintenance package to Atmosphere. :slight_smile: I tried npm first, and kept getting errors with regards to importing meteor/mongo.

1 Like