Is packages-for-everything the solution to all Meteor problems?

packages
1

I’m new to Meteor and am trying not to bother anyone with FAQs, but after extensive Googling I felt a need to clarify some points.

First of all, I was somewhat taken aback by the use of the global namespace in a modern framework full of 3rd party packages just waiting to conflict with it. The testing system (Velocity) is just coming together now, and I wonder how DI will be achieved. But presently I simply have to get things working in the first place, and file loading order issues have cropped up. Putting all my globals into lib/_namespaces.js is brittle given the load order rules (you have “main” as loading last but no reliable rule for loading first).

I decided to use the Iron:CLI system with its router and “scaffolds” as a standardized way of structuring the app (suggestions to “just refactor your directory structure as you go” don’t sit well with me…frameworks are about leveraging the experience of others no?). Correctly it makes individual files for each collection. But combining this with aldeed’s autoform package and its reliance on a global Schemas object presented the load order problem. It seems aldeed suggests just throwing all the Schemas for all the Collections into a giant “common.js” file which clearly will get unmanageable for a production app after a few hours of coding.

I read somewhat wandering proposals about namespacing using lazy resolution and then terse suggestions about using packages that seem to make more sense. There are a few rarely used packages on Atmosphere concerning namespacing, nothing promising. So I went ahead and learned how to make my own package and exported the Schemas global from that. Seems to work so far, but wow—there’s an uncanny valley between cowboy-style freecoding with globals and using the package system to impose some degree of structure. A learning lurch.

The next investigation was just what was required in practice to realize the write-once-run-anywhere promise with desktop and Cordova mobile apps running on the same reactive codebase. I figured there must be cross-platform HTML5 frameworks that cover Web, Android and iOS in one go, each with their native standards. Sadly no luck…I found Bootcards to be in a primitive state, and while Ionic does Android and iOS in one go, it doesn’t do desktop web at all. Ratchet, ironically, was developed by largely the same team behind Bootstrap, but they inexplicably made the two incompatible. With this mess I would probably be best off implementing individual mobile apps (perhaps in Framework 7 and Materialize) even though most Meteor stuff is expecting Bootstrap. But how to avoid DRY violations and maintenance headaches that would come without a common codebase? Isn’t that the point?

SO pointed to this example which selectively exports from packages to desktop and Cordova. This site comes to the same conclusion. Any app of any size, I would say anything beyond a day’s sketch, seems to really need to be factored into packages from the get-go with Meteor as it’s designed right now. It’s not taught that way in any of the main materials I could find, they’re all just “Dive in cowboy-style, go ahead and repeat yourself, Mongo denorms everything anyway, etc.”

Is this the consensus among the experts then? Packages-for-everything is the best and only solution to building anything production-capable in Meteor? Is that going to be the case moving forward? Am I asking too much of this framework too early in its evolution?

19 Likes
2

In my experience thus far with Meteor (going on 6 months now), you seem to be absolutely correct in your thoughts of “packages for everything”. I’m in the process of writing a large-scale app and the only way to keep my codebase manageably small is to continually take the reused components and separate the code into packages. This does seem to do a good job of separating functionality, but it can be a pain at times. I don’t know that it’s going to get any better as the framework evolves, because the package system is rather robust and useful.

I think of it kind of like Python - there is a library for EVERYTHING and I imagine that as Meteor gains popularity, the package system will get better, but I really think that’s going to be the way that the code stays modular. As it is, the packaging system does need to get better as far as its learning curve goes, but I think it’s getting there.

3 Likes
3

Fantastic question! It’s something I’ve struggled with, too. And I’m curious to see all the answers.
I tried both and, at least how my mind works, I find the “package all the things!” approach to be less organized. I like to see all my collections in a single folder, all my server-only code in a server folder, etc. In my opinion, it comes down to deep vs. wide.

The Wide Approach (Packages)
Each package is a swath including everything you need for that module. This sounds very modular & I can see why a bunch of folks opt for it. Unfortunately in my projects, relationships exist. You have server methods that update 2 collections at the same time, or template events that modify other templates. I could never find a clean way to modularize this, so with my tail between my legs, I gave up on 90% of my packages. If I can’t use it for another project, it doesn’t deserve to be a package IMHO.

The Deep Approach (Standard)
I respectfully disagree that the load order is brittle (albeit it is a little confusing). It loads lib folders (deepest first). Then, it loads regular folders (deepest first). The load order is alphabetical.
My lib/_namespace.js looks like this:

R = {}; //Global
R.C = {}; //Collections
R.Cons = {}; //Constants
R.Fx = {}; //Functions
R.S = {}; //Schemas

if (Meteor.isClient) {
  R.A = {}; //Animations
  R.D = {}; //Dependencies
  R.T = {}; //Current parent template
  R.T.RD = new ReactiveDict();
}

Now I can share things across my entire app without polluting the global namespace. Plus, typing R.C. in the browser console & seeing ALL my collections pop up? That’s just fun.

Front-end frameworks
Your second point regarding mobile isn’t really meteor specific, but a great one nonetheless. I’d say Meteor/Cordova get you 70% of the way there, but there’s no one-size-fits-all when you have to consider hardware configs (screen size, pixel density, bluetooth, gps) and OS configs (modem, address book, other apps). These things aren’t escapable. Heck, even the Google Maps API has iOS-specific calls when it comes to saving places.

Additionally, you have to ask yourself if an app can look the same on all devices. In my opinion, there is nothing wrong using Google’s material design on web, iOS, and android. Or, conversely, Apple’s Human Interface Guidelines on an android. The point is Meteor/Cordova gives you the choice, which is neat.

7 Likes
4

`Thanks very much for the replies so far. Some more points:

  • A major motivation for using packages for Cordova/mobile apps is, if you don’t, then Meteor sends all your desktop-focused code to the mobile apps anyway by default. There’s no other way apparently to stop it from doing so (is there?). This means the mobile apps will get hot code reloads for desktop changes, slowing things down a lot (OK there is a technique for disabling hot code reload on mobile apps until the app is suspended, but that means long startup times which sucks for an app).

  • Given that fact, and given the fact that I really want target-specific UI frameworks (Apple will not promote an app on its app store that looks like an android app yo), I either have to use packages with all the organizational overhead, or Meteor could develop a more thorough targeting system for project directories. That way, in addition to client/ and server/ directories, there could also be client/cordova/ for shared Cordova code, and definable target directories such as client/cordova/iOS/ and client/cordova/android/ and client/browser/ that would only provide their code and templates to their defined target. Ideally these would have an override logic, such that templates could be shared from client/ but overridden selectively for each target as necessary. I’m not sure how packages would work to do this sort of thing now…?

  • Because you can nest directories and the deepness counts in load order, it is brittle to use lib/_namespaces.js currently. I would recommend Meteor adds a rule that anoints a certain file path as loading first, and if people have been using lib/_namespaces.js they may as well choose that one.

  • At some point we might decide it’s time for Meteor to “grow up” and adopt namespaces as standard, and naturally I think that time is now. So all the training materials should be rewritten to use app-specific and package-specific namespaces for all their global declarations, with every app and package having its own com.example.app.* or com.example.package.* namespace standard (actually the “Meteor” way of saying that might be username.appname. e.g. aldeed.autoform.Schemas). Maybe I’m having a bit of culture shock coming from type-safe systems like Scala and have to get used to riding cowboy-style on the JavaScript Wild West. But the example of aldeed:autoform expecting you to use a global Schemas object but not bothering to declare one in its own namespace and smacking the unsuspecting user with file load order problems is instructive. There are benefits to civilization.

Anyway I would like to hear more of your insights because I prefer to do things right the first time when possible rather than continuously refactoring especially on a project-wide scope.

1 Like
5

ES6 Modules are going to be an amazing addition to Meteor (without Meteor having to do anything at all).

It means you’ll be able to use System.register to load an entry point, and all your code can be stored in the public folder (or served via server-side routes for security depending on logged in user). Then since all of the Meteor globals will be available in the app already, you can simply load your ES6 modules and start executing them in the app, instead of shipping them all at once pre-compiled. You’ll be able to bundle things for production and serve certain bundles based on routes, etc.

I can’t wait till this day.

4 Likes
6

I think that for the moment, packages-for-everything is the only viable solution.
It’s not that easy, but it works once you have understood how everything work.
Main problem for me ATM is that all packages lies in a flat folder, which is not very good for organization.

3 Likes
7

I’ve posted a theoretical way to use a module loader (Browserify, etc) here. It shouldn’t be too hard. Just make your bundle load after everything else. EDIT: I did this with webpack.

1 Like
8

Packages for everything is a pretty good way to start.

I made the following to lower the barrier to getting a package started by writing boilerplate for you: http://package-kitchen.meteor.com

The load order behavior of Meteor was one of the first things that drove me to prefer packages. The control over names/exports is another.

That said, Meteor is a different ecosystem than Python or Java, and things are going to work differently. As a member of the community, you can work to champion proposals that make things more like you’d like.

5 Likes
9

@deanius very cool site! I think that’s a very nice way of doing things.

A cool thing to do, would be to add an interface where someone could upload js files that could be automatically sorted into a directory structure. Are you open to pull requests/is this something you want to further develop? I could see this as something that could eventually be integrated straight into atmospherejs to help people get started developing packages.

1 Like
10

Yes, i’m quite open to developing this idea :smile: One hope for it is that you could use it inside your app, much like Velocity, to create packages in-place as you develop it. And, eventually, if it would help users on the publishing side too…

I came from Rails where people thought Gems were only for other people’s stuff. As a result, people didn’t tend to Gemify apps as they wrote them. The end result of this were apps with folders holding dozens of files, and features which seemed to more-than-linearly degrade the time it took to run the test suite.

Anything we can do to make the on-boarding to packages a simple, beginner-level concept, will help to keep the ‘typical’ Meteor app as simple as it can be - just a little code to glue together the packages that make up its features.

Let me know what else you want to do with this @jchristman !

2 Likes
11

I can’t wait for this too!

12

Thanks for this excellent discussion. I started out replying here, but eventually created a new thread instead. I’m curious to hear the opinions of folks like @jchristman, @mattkrick, @vjau, @trusktr, @deanius, and so on tackle testing in this app as packages paradigm. Specifically, how to disable stuff like background jobs while the tests are running.

1 Like
13

Would you guys suggest this even if you were going to use React?

1 Like
14

With React you could use universe:modules.

1 Like
15

Thanks for that @mattkrick, I agree the deep approach seems best to me. Any chance of you posting a github repo with your folder/file layout and concrete examples of using these namespaces you mentioned?

Thanks!

1 Like
16

Very slightly off-topic: Atmosphere should be taken out back and shot. I avoided learning to roll packages myself for a long time - “why bother?” - till I ran into so many problems so many times that I started googling enough to find threads like this.

For the packages that I use:

  • most are not current, many badly, where the original publisher has moved on.
  • most are posted by third parties, who have no connection with it and post their package.js source nowhere. About half use account names that impersonate the base project, but in many/most cases actually have no connection with it.
  • the version of the base project is unclear, and can only be found (if lucky) by digging the source out of .meteor/local.
  • dependencies are often incomplete, naturally, since I’m mixing things together. But without the original package.js, one has to start from scratch in creating a correct dependency list.

Finally, worst of all, this is unlikely to be fixed without changes to the site mechanics. At the barest minimum, it MUST be possible to contact the publisher of a package - the maintainer of the package.js et al. Without this, someone who might otherwise republish a package to bring it up to date is unlikely to; no one likes unnecessary duplication if the original publisher is going to update a package “tomorrow”.

1 Like