Meteor 1.3 *early beta* now available

announce
61

Outstanding improvements! Eagerly waiting for stable release.

62

Can I import from specific files in other packages, like:
`import {Posts} from “my-modular-package/collections” ?

1 Like
63

I think the interface should be:

  1. autoimport package like insecure and autopublish as suggested by @luisherranz:
  1. when autoimport is removed, a main.server.js file is generated which solves the following problem mentioned by @benjamn:

It would include ddp etc and be the equivalent of the following extremely familiar code:

var express = require('express');
var app = express();

app.get('/', function (req, res) {
  res.send('Hello World!');
});

var server = app.listen(3000, function () {
  var host = server.address().address;
  var port = server.address().port;
});

This would bridge the gap to the Node world. The end goal would be that the app could run on NPM without using the meteor command. I don’t know all the implications and implementation challenges if any, but even if it couldn’t currently run on NPM, it would pave the way in that direction. It would be a good marketing move to blend in with the rest of NPM. I think it’s important for non-meteor developers to be able to look at Meteor and immediately see something that gives the semblance of something they are familiar with. It would also give advanced developers the opportunity to customize something they couldn’t before. And of course it would solve the problem of having all the standard packages loaded in the correct order on startup. Many frameworks generate this sort of entry script for you to modify, so it’s hardly irregular.

Lastly, if the autoimport package was added back, this would automatically be commented out or ignored.

  1. Included packages behave like this:

LEGACY ATMOSPHERE PACKAGES:

  • without autoimport package: are given a chance to run files in specified sequence without exporting any variables into global scope. The purpose here is to make sure things like the override of built-ins still occur, but without forcing anything new into global scope. However, the interaction between one package to another would continue with the same contract its always had.
  • /w autoimport package: like they currently do.

NPM PACKAGES:

  • without autoimport package: just as you would expect, no special behavior.
  • /w autoimport package: if a meteor-onstartup script is specified in package.json it will run any special code that needs to happen on startup such as built-in overrides, global exports, adding blaze templates, etc. api.export would need to exist in this script, but the whole thing would run in the regular package scope so built-in overrides could be applied to any used sub-packages. There would be a meteor-ontest-startup possibility as well. Meteor-specific package.js files would obviously not be needed anymore–we are now a pure NPM package! The main idea here is A) to solve redundancy: if you import and export all your code properly from the files themselves, you shouldn’t have to export anything from package.js or set the file load order; while preserving something Meteor package already rely on: B) running code on startup that does things like override built-ins (but in an NPM-idiomatic way via scripts).

This will delegate the non-standard stuff we allow to seep through the current package system to happen in an optional smaller confined place than before, and without requiring developers to make packages the old Atmosphere way. Like, if you really wanna do that fancy automated/implicit stuff within an NPM package, you can, but besides the special script you will have to make your package look and behave like a regular NPM package. We are now “guiding” developers into the “pit of NPM package success.”

Ideally, the meteor-onstartup script wouldn’t need to exist (just as if autoimport wasn’t included), and package developers instruct application developers in what to do to get their code running (the choice will be theirs), but obviously it’s of the utmost importance we are backwards compatible, especially as package developers migrate to the NPM style while application developers remain in the Atmosphere style–that’s the problem we are trying to solve here; and I also think it remains extremely important that things are as easy and automated as possible for novice developers; even advanced ones–if we can continue to provide the option that, for example, the meteor collection hooks package automatically makes some overrides to your collections, we should. I personally don’t like to be told what needs to be implicit and what needs to be explicit, the decision should be mine–we need to add the explicit option, not simultaneously get rid of the implicit option we are already the best in the business at offering.

Interestingly, I suspect this to be the most important and most common use case, as most application developers will likely stick with the implicit package loading option for a long time to come, while package developers (likely more advanced developers) will want to switch to the more advanced pure NPM import/export approach. That’s why it’s so important we consider and get this use case right.

Another important thing to note is this: the same NPM package can be used with autoimport on as we all as off. Without autoimport, application developers must follow the directions to import the file and call the code that will run onStartup routines, whereas application developers with autoimport included will have the meteor-onstartup script for the given package handle that for them. In that way, the new NPM packages we are guiding package developers to create will service both use cases while otherwise looking like a standard NPM package. Slam dunk!

  1. THE CLINCHER: a solution for partial lazy-loading should be a first class priority so developers can migrate/upgrade their app incrementally or simply be able to take advantage of file-order-control where they need:

to accomplish this, rather than having an imports folder, statically detect which files are ever imported, and automatically remove them from the list of files “eagerly” loaded. This would only be a behavior that occurs when the autoimport package is still included. It would greatly ease the path towards removing the autoimport package altogether, i.e. after you manually imported all files ever used.

In addition, this would nicely solve the confusion discussed by @serkandurusoy and @SkinnyGeek1010 above:

  1. Without the autoimport package, developers manually add any assets, templates, css files, etc contained within packages. There’s just gonna have to be some sort of Meteor configuration file at the application level that starts to become common practice to use: meteor.config.js. In it you can specify which assets from which packages are sent to the client. Ultimately this should evolve 2 directions: A) Webpack’s feature to include assets within js files which become part of chunks, and B) the following idea where Meteor provides a router in the core, which efficiently doubles as a way to specify how code splitting occurs across multiple entry points:

  2. FUTURE: I know webpack and code-splitting is for a post 1.3 release, but it seems to me that if MDG brought a router into core and coupled the router with custom webpack-like code-splitting features, we could greatly simplify the Webpack interface and make it feel more natural.

Why/How? Description of chunks emitted would live in the same location as the route definition, no config file other than route definitions needed. As for require.ensure, we’d implement that separately (and again on our own, not using Webpack under the hood). When you rely less on routes in single-page apps, code splitting becomes all about require.ensure anyway.

How/Why? The common code is loaded based on what’s in main.client.js and only a second request is needed wherever you wish (e.g. a route, or upon routeless action) to ensure you have anything else you need, while of course achieving the goal of never loading more than you need until you need it.

If we simply had require.ensure without even “lazy entry point routes” we’d have already done 90% of the job. I personally am a lot less interested in defining entry points than just being able to load a batch of code/chunks manually when I need it, but I also assume lots of people would prefer the route-based approach as well since it would allow for a single request on page load. Starting with just require.ensure could be a great feature to do in a 1.3.1 to phase the development of all this.

Perhaps, we never need a code splitting config file (i hate that F#!ing file), and all our chunking is done within code using require.ensure, as well as including any assets (yes, via javascript) that need to be sent to the client within that chunk. This implicit automated approach would mean one less concept Meteor developers have to learn (for the most part). It’s a lot easier to explain just use require.ensure. In addition, what I think might make a lot of sense is instead of specifying your code in the callback to ensure, utilize something like a decorator (almost like “use strict”) at the top of a function to annotate specific exported functions as being the starting point of a given chunk:*

FlowRouter.route('/blog/:postId', {
    action: function(params, queryParams) {
        @chunk(['additional', 'dependencies'])
        import {post, comment} from "lib/model";
        //etc
    }
});

That would create a chunk according to similar rules as Webpack containing “additional” and “dependencies” chunks, and the “lib/model” module. The transpilation result would be the contents of action function being passed as the callback to ensure. This would save us the unnatural nesting that comes along with ensure — it’s a seemingly small thing, but would give code-splitting a magically harmless feel, and because of that frictionless would lead to an explosion in its usage. If we could find a standards-based way to achieve something like that decorator instead of having to use a callback, I think we’d be on to something. It would be a very simple and special interface to be sure.

Up until that route, Meteor would have only loaded the common chunks/modules. If it’s the first route, Meteor could just make requests for 2 files. I’d be fine with that. If the router was added to the core by MDG, well it could likely figure out how to make that first request load only one javascript file. In either case, unlike webpack, we wouldn’t need complicated “webpack.config.js” files. Transpilation and building would be a totally separate affair in our case, currently handled by build plugins. The biggest concern MDG has had with Webpack’s interface is all the extra advanced work it expects of developers to configure. It’s too much boilerplate. That’s not the Meteor way, and I wholeheartedly agree that’s a good decision. I hate all that boilerplate config. Yuck! Lastly, I’d like to reiterate that by not having to pass all your code in a callback to ensure we haven’t disrupted the hierarchy of your code — we have let it remain flat, which is awesome! Otherwise, importing and exporting is business as usual, not caring if you are doing it asynchronously in chunks.

As for arguments against duplicating what Webpack does and rolling our own, my response would be: code splitting is the best of Webpack, their interface is poor, and we already have other ways to perform the duties of loaders, i.e. the Build Plugin API–so let’s focus on just getting code-splitting right. Introducing the webpack configuration into Meteor or any app is a lot of extra boilerplate; and since Webpack does a lot more than just code splitting is too advanced and cumbersome for even intermediate developers. My hunch is that a way better automated interface could be developed, definitely if only code-splitting was addressed. I also feel like building for the client is so important it should be something part of ES7 or ES8. I know that’s farther outside the realm of typical language currently features being tacked on (since it deals both with the server and the client), but code-splitting is only going to become a bigger and bigger problem as the web progresses and our apps get even bigger. As I was saying above, wouldn’t it be a lot simpler if entry point bundle configuration happened in the router–there seems like there are options to implicitly do this and keep transpilation of various file types separate.

Meteor, being the only true option in town when it comes to reactive universal apps, might be the perfect guys to pioneer a format/interface for code-splitting and the other isomorphic package implementations for NPM discussed above. I.e. a set of tools for NPM that exist outside of Meteor. Unlike Blaze, which was barely supported outside of Meteor, I think in this particular case–given Meteor being a longtime pioneer of the isomorphic web and knowing it better than anyone–we may be able to offer something to the greater NPM community that they actually wanna use, especially if we promote it properly (like we never did with Blaze).

It seems we are at a place–thanks to @benjamn 's amazing work–where the foundation import/export functionality is done, and now we need to flush out the interface. I know all of it won’t get done in 1.3, but hopefully we can have a nice roadmap of how we’d like to evolve the interface here. WE NEED TO THINK ABOUT THE END GAME HERE TO GET THE NEXT STEPS RIGHT. That’s my main message.

What I’ve presented here is one possible end game. I’m sure I’ve missed some things and misunderstood some things–let it be a template for thinking of the bigger picture we want for Meteor. To sum it up, from my perspective, we don’t want only explicit (and certainly not only implicit, or we wouldn’t be having this conversation today). We would like both implicit, explicit, and more importantly a way to gracefully upgrade from implicit to explicit or simply maintain partial implicit + explicit apps. To do this we need to tackle the resulting matrix of adding the fact that packages may also be developed along this spectrum. And to top it off, we need to at least know our “webpack strategy”, which I’m voting should be a custom version all our own focusing just on code-splitting (I’m pretty sure I read somewhere that @benjamn is thinking the same).

That’s all she wrote. …@benjamn , curious what you think about the idea of removing any lazily loaded packages from the set of eagerly loaded packages via automatic static detection of whether one its modules is imported??

7 Likes
64

@faceyspacey Are you a real person or a Discourse post length edge case test function gone wild?

32 Likes
65

In that case it would have been Jeff’ed a long time ago.

1 Like
66

In the doc you gave an example of importing a package by NOT using a relative syntax. So what is your search order in that case (given that you have to search packages and node modules…)?

@tarmes this ASCII-art directory tree illustrates how Meteor packages are installed in bundles. In short, Meteor packages are treated much like “global” node_modules, in the sense that they can be imported without relative paths from anywhere in app code, and they can also be shadowed/overridden if the app installs Node modules locally (in /app/node_modules or /app/client/node_modules or wherever). Given this directory structure, the lookup rules are the same as the ones used by Node.

In the not-too-distant future, I foresee more and more Meteor packages simply becoming npm packages that apps can install locally, so treating them like npm packages now is a step in the right direction, not just an implementation trick.

1 Like
67

@benjamn in your presentation: http://benjamn.github.io/empirenode-2015 , last slide mention http://rollupjs.org/ Is any plan to integrate this bundler beside webpack on meteor ??

2 Likes
68

I made a small github repo that shows some basics of using this new import/require syntax with npm react components. Might come in handy for somone so here it is.

This app uses the react component formsy and integrates decently with aldeed:simple-schema.

2 Likes
69

I feel the same way. Webpack is hard to configure and could be easily made simplier. However, where I disagree with you is: we should re-code the entire thing in the Meteor build process. It’s a though I was entertaining too until I decided to check how this can be done.

Be ready for days and days of headache just to understand how Webpack is working behind the scene. If you think Webpack config are complex, this is a very complex engineering problem that has been solved very well. I think it would be an incredibly big job (and quite frankly a waste of time) to try to do EXACTLY the same thing just to get a simplier way of using it.

Instead, why not making Webpack easier to use? Why not find a better API Meteor users could use to simplify the integration of Webpack with their projects?

What if integrating Webpack was not a matter of writing a “complex” config file (quite frankly it’s not when you’re use to). What if all you needed was to add the few packages that you needed? Example:

meteor add webpack:webpack
meteor add webpack:jsx-loader
meteor add webpack:sass-loader

And use the exact same structure than Meteor 1.3.

When the gap between Meteor and Webpack is that small, what’s the point of reinventing the wheel?

Beginners don’t need to opt-in as soon as they learn but they could just by adding a few packages. What do you think?

14 Likes
70

Hey @faceyspacey I challenge you to put a max 100 Words/600 Characters tl;dr at the beginning of your posts.
FYI: That post has 2528 words & 15170 characters. At avg. tech reading wpm that is 15-20 min. Without replying. :flushed:

So: I love your posts (they are full of great ideas :+1:) but often there’s no time to read it all (esp. on mobile) and I believe many people, incl. MDG Employees, feel the same way. A gist is great to navigate… I already feel bad not to be able to read it all due to work, real life, projects etc. Don’t have your contributions fall under the table because it is too hard to stay on-top.

// off-topic over

24 Likes
71

This sounds amazing @benoitt ! I think this coupled with whatever folder structure MDG settles on (in their guides) for 1.3 would make for a killer setup. Having a default config with easy loader support solved 80% of use cases.

How would you handle builds?

1 Like
72

@jeremybyu I think you should post the “gotchas” over at the meteor issue tracker so that they reach the right audience and get resolved.

1 Like
73

I posted here, the pull branch for this change. Hope this helps.

1 Like
74

@benjamn Is there (or will there be) some API we can use to load modules over HTTP (or HTTP/2) at will? SystemJS has this so that it’s possible to load an entry point when you say so (and the entry point can use any of AMD, CJS, or ES2015 module formats). SystemJS will then know how to fetch all dependencies for that entry point (they can be separate files, all stuffed through a single HTTP/2 connection).

3 Likes
75

Yes, I’m also interested in that to create a plugin platform for Wekan. These plugins would be loaded dynamically at runtime depending on the current board.

1 Like
76

they’ve gone out of their way to make sure they’re not just supporting straight up ES6 modules but also all backwards compatibility by supporting the old globals and load order, it sounds like you don’t have anything to worry about

3 Likes
77

they specifically say “lazy evaluation” to avoid that mishap. when I first read that I thought the same and almost jumped with excitement until I realized it’s just evaluation :disappointed:

1 Like
78

I really like the idea of making Webpack stupid simple and hiding it behind some “Meteor magic”, but I imagine a different approach to make it stupid simple for beginners.

If Meteor included a router in its core (and it really should!), or a common API for community routers to interact with, we could follow a convention for defining routes that includes the routing info and the entry points for that route. I’ve always liked the client/ server/ both/ directory convention because it forces familiarity and structure, which IMO Meteor needs more of. Along those same lines, if we had a router in core we could define routes with something like this

// routes/login.js
export default {
  route: '/login',
  title: 'Log In',
  entry: '../client/components/login.jsx',
  styles: ['../client/styles/a.css', '../client/styles/b.css']
  authenticated: false,
  lazyLoad: true; //optional
}

// routes/home.js
export default {
  route: '/home',
  title: 'Home Page',
  entry: '../client/components/home/entry.jsx',
  authenticated: true,   //this could use the core accounts package, maybe 
                         //allow the user to define their own auth function here
                         //too if they use some other auth solution
  failedAuthenticationRoute: '/login'
  lazyLoad: true;
}
```

That way a router in the core could just require everything in the `/routes/...` directory and create a definition for each route based off of that. The build system could then use the same `/routes/...` directory to generate a bundle for each route by looking at what the `entry` file imports. Whether it used SystemJS or Webpack in the background wouldn't matter, and something like this would keep in line with the "Meteor" way of making advanced development features easy for complete noobs. It could also open up the possibility to server side rendering pretty easily. Dynamically generated routes would require some API to achieve the same affect.

This could be kept in a package so you could just do `meteor add routes`  and you'd be set. I'd be one happy camper if we had something like that.
1 Like
79

when will 1.3 be released? is it ok to use 1.3 features to develop new app right now?

80

I imagine you should be prepared to potentially make major changes as the release is being actively developed.

2 Likes