Oooooooh. Now that’s an excellent utility. Definitely going to add that to the Atom toolkit. It may already have half of the refactor/move functionality, since it’s already building up an internal map of all the import statements. Thank you, thank you for your report from-the-field that this is working.
I also just found this one:
That’s what I suspected. I’ll hand it to WebStorm… they have the better refactoring tools; and are doing their best to be the VisualStudio for JavaScript. The problem is that I know how to hack on Atom, but not on WebStorm. And I specifically moved away from WebStorm so that when I needed to hack on the editor itself, I could. I may renew my license though, just to check out it’s refactoring utilities for ES6
It has come up. The people who were unhappy with imports moved to a product called Meteor which managed to implement a Node based platform that didn’t involve require and import statements.
There’s massive selection bias and cognitive bias happening in looking to “the rest of the node community” on topics like this.
Exactly. Selection bias at work.
Regardless, I think this has come up before; and there are tools that exist to address this issue. I just don’t know which ones they are; and they haven’t made their way to Atom quite yet.
@sashko the problem with that train of thought is that is from the perspective of Node users, and not Meteor users.
Keep in mind that Meteor was built around NOT using import statements, and the methodology & part of what attracted users to Meteor was the fact that you did not have to spend time on worrying about those and could instead spent time on developing your app.
After C# and Django, using Meteor and NOT having to import, was a breath of fresh air. It’s one of the things I loved about working in Meteor.
So while Node users may be used to it and happy that way, Meteor users would likely not be. And considering some MDG employees have openly stated that they recognize Meteor has became more complex to work with, and will be receiving another pass at becoming more user friendly again, this is one of the top areas that should be addressed.
If a Meteor user from the beginning were to start with 1.3 onward, having to add imports to their older projects is easily the most jarring aspect of the new Meteor, and likely the least user friendly aspect of Meteor as well.
@Peppe_LG - the optional is only supposed to be temporary for backwards compatibility, and they said it will likely be required in the future.
JetBrains IDE’s other than WebStorm have support for automatically adding imports where needed , renaming files, etc. (although WebStorms Meteor support has not been upgraded with any of these features yet). It should not be very hard at all to add similar logic in Meteor to help assist with users adding imports to their projects.
I, for one, am dreading the day that I have to go through hundreds/thousands of files in my large project and convert them over to the new system. And I do not like increased development time on new projects. This is easily my biggest area of discontent with Meteor as it stands right now.
I truly believe if Meteor is going to make imports required, there should be a system in place that helps legacy users of Meteor, because if not they are basically being punished for starting earlier with Meteor & needing to put in a huge amount of development time just to be able to keep Meteor up to date.
Not to mention, one of MDG employees on here said something along the lines of “We have some things planned for this and we think you will be happy in the end” - so I truly do hope this area is addressed.
Webstorm is halfway there. After discovering last week that I did not know as much about imports as I thought I did, I began tinkering with file organization on an app I’m currently working on.
I must admit, I’m not happy with the results and miss working without them dearly. I know, it is optional etc, but one tends to try and move forward so that they don’t suddenly get left behind on a breaking change.
Anyway, webstorm can refactor moving files, but drag-drop and refactor > move yield different results, the latter being the better choice.
Furthermore, refactor > move does update other files that reference the moved one but the moved one does not update its own references, which is a huge gotcha!
Another gotcha is the problem of relative vs absolute paths. Absolute paths don’t get resolved properly so referencing across modules (read: top level directories that contextually divide the app, eg api/ui/startup/utils…) yield import paths that often look like '../../../../../api/etc' which make it much harder to understand the code. The everything is a global so I am using sensible namespaces approach was much better because it already carried a cognitive baseline to undertanding an app’s structure.
Also, you just simply can’t divide a file into two different files once it grows. You have to track down all the imports and update them manually.
Finally, re-exporting from imports, either directly using export from './something' or import { foo } from 'bar'; export {foo}; causes webstorm to loose the ability to actually resolve the contents of foo such that anything attached to foo becomes hidden away, whereas, directly importing it from its original export would yield in autocomplete assistance.
Anyway, from what I gather, one needs to really plan long and hard about the app structure and file naming schemes, otherwise, refactoring becomes a mess.
Now here’s what I wish we had:
Meteor’s familiar globals
A feature that allows wildcard adding of files withing packages (just like the root app)
Ability to use imports without having to place the imported file in the imports folder, such that it is merely a helper/override that mitigates edge-case load order issues.
If I at least knew imports would never ever become the default and old Meteor load order rules would stay there forever, I’d probably never switch to imports.
@awatson1978 Thanks for bringing this up! I’m currently migrating a relatively small codebase from the 1.2 client/server folder structure to imports and the api/ui strategy recommended in the guide, and have found it to be difficult also. (Can’t imagine trying the same on a large, mature project…)
To follow up on what you’ve said – re-organizing the folder structure in 1.2 was amazing. When you realized that you wanted to split a large file up, create subfolders for complex modules, etc., that was as easy as using a file explorer. Not so much anymore.
Also, I’m sure I’m not the only meteor dev who has experienced the frustration of wondering why an event wasn’t firing only to realize that the .js file containing the event wasn’t in the import chain.
That being said, the pain of folder refactoring has definitely lead me to think more concretely about naming/structure conventions. I’ve realized quickly that I’d rather pay now than pay later when it comes to reworking my import statements.
Moving a file and having to manually address the existing import references just feels like a stone age user experience, but ultimately I think it comes down to a question of ownership of a pain point.
Should MDG own this as a way to improve the meteor dev experience/migration path? Or should the community own it as a way to improve IDEs/toolsets?
Yeah, especially if you split up your app in multiple packages, it becomes very hard to move files around. And no editor knows what to do with ‘meteor/…’ imports. For packages you publish publicly it becomes almost impossible to refactor.
I do remember from my Java days that Eclipse had a shortcut to automatically organize all the imports which worked great, but that’s of course much easier with a statically typed language.
What I currently do is only use absolute paths everywhere, so at least I can do an easy find-and-replace.
I think @sashko isn’t necessarily just saying node, Meteor is following JavaScript as a whole. Which is fantastic. Aligning Meteor with mainstream JS is the most exciting thing to happen to Meteor since it’s release, now it’s finally a solid platform that I can bring other devs onto in a matter of minutes. Previously having to explain how all this “magic” happens was absolute hell, it doesn’t make sense.
I also disagree quite strongly with people saying imports make developing slower, while it may take you an extract 10 seconds to write your import statements, the long-term benefits knowing where things are to debug accordingly will save you far more time. Very much along the same lines as writing unit tests as you develop, takes longer, but will save you days in the end.
I do understand people are scared of change and 1.3 was a big one, but if you’re an experienced developer it was amazing. For the hobbyist or junior dev, maybe less so, but I don’t think MDG’s future is building a platform for them, it’s building a world-class platform that’s suited to larger projects.
If you structure your app well, then imports should be a very minor issue. Mantra examples have imports nailed down very well. Split your app up into ‘module’ folders, each with an index.js that gets referenced. Have a configs folder at the root of your app, which you can reference easily, so if you change something somewhere, you change it in one place.
Eg:
// configs/components.js
import MainLayout from '/client/modules/core/components/main_layout.jsx';
// or '../modules/core/components/main_layout.jsx';
export {
MainLayout
};
// Rest of my app:
import { MainLayout } from '/client/configs/components';
This makes moving files around far easier, since you don’t have to update references app wide.
Then within each ‘Module’ you simply do
import ComponentName from './components/component_name';
Alternatively you could make an index.js in /components/ too and do
import {Component1, Component2, ComponentName} from './components';
This all depends on the style of app you’re building, but once you’re used to it, it’s a very simple and logical system. The ‘magic’ of auto things is great when first starting out, but unrealistic in production.
Anyway, just my thoughts on this topic
Side note: I do understand the frustration of junior devs, I was one myself, as we all were. I do think a solid beginner friendly guide/tutorial should be made, whether this is from MDG or the community. However instead of doing the usual ‘LOOK HOW EASY THIS IS’ it should be a realistic guide, on how to build a good first app. Using imports, using correct folder structure. I personally never liked how the guides used the if (Meteor.isClient) {} stuff, I found that very unhelpful as it’s realistically not what you’re ever going to use in prod. Teaching beginners the wrong way isn’t going to help them.
I don’t think that anyone is saying that imports inherently make development slower. Rather, the way that imports are handled in Meteor, coupled with the fact that no IDEs really have out-of-the-box solutions for updating imports on file movements make restructuring a 1.3 app tougher than it should be.
To follow up on your junior dev point – I agree that teaching beginners the right way is something every tut should strive for. But learning Meteor from scratch was tough enough back when Discover Meteor was the go-to starting point. Knowing what triggers a template render, pub/subs, allow/deny vs. methods – a lot of these concepts are relatively new if you haven’t worked with reactive data before. Throw in modules and imports (and the fact that there really isn’t a de facto Discover Meteor 2 yet, and I think things are even tougher than before if you were to pick up Meteor today as a newbie.
Yeah, but it’s like we’ve all been eating bacon everyday for the past few years and now we need to start eating vegetables. We all know and agree that it’s better for us in the long term, but it’s very tempting to keep eating bacon. And I think this analogy holds for a lot of what Meteor is doing right now, like Apollo vs the current livequery system.
Yeah as I said it’s harder for beginners, I know that. However for me, I had to get two other devs up to speed on my project, them knowing JavaScript, it was easy, they didn’t have to know Meteor. When 1.2 was around it took me a day to get someone up to speed with how everything worked and all the little details of Meteor, like the nested folders for build order. So I guess it’s an interesting one, experienced devs it’s probably easier than ever to start, beginners its harder than ever.
@joshig As you say it may not take long to write import statements, but keep in mind this is not Meteor’s traditional method. So what about the projects with thousands of files that do not have import statements? How much time do you think that will take? Might there be some issues with this in the future when backwards compatibility gets dropped, where developers would try to go as long as they possibly could before they do the upgrade, to avoid the lengthy job of upgrading a huge project to a new import system & folder format? What about when an update with vital security updates or something along those lines is released?
I think it’s great that they are building a world class platform. I do not think it’s great if they “sell the soul” of Meteor in order to become suited for larger projects. Meteor was built around fundamentals which are the reason most of us are here. Why should it even be an issue that some of us want Meteor’s future to be built around those same fundamentals as well? If you take those away, it’s not really Meteor anymore.
Especially on an issue such as this, where many projects (even my largest one) doesn’t actually require it and will see little to no gain from it? When adding support for “the old way” shouldn’t really be too difficult of an issue, as functionality similar to it is commonplace for plugin’s in some major IDE’s anyway.
And regarding getting devs up to speed… My job actually hired some frontend devs. They were somewhat familiar with javascript, but not specialized in javascript. This would have been completely sufficient pre-1.3 - and it’s a shame because I instructed them to put that as the requirement on applicant’s resume’s specifically because of Meteor’s state in 1.2…
But they started about a month after 1.3 dropped. The official tutorials also got much more difficult at the time, and even the folder structure was completely different from our huge project that they were going to assist me on. So I had to teach them Meteor (the way our work project uses it) from the ground up myself. It was a very unfortunate inconvenience that things changed that way after they were accepted for the position and right before they started.
I personally hope the day does not come where I have to go through the whole project and spend all the time restructuring it manually, for features that do not benefit the project. How do you think that will look to management in plans/progress reports? Definitely not time well-spent. So I am sure it will run in to that problem I mentioned earlier, where it will be pushed back as much as possible.
I do agree it’s a problem if you’re trying to update from 1.2 to 1.3, but my guess is from MDG’s standpoint, that’s a very short timeframe problem. It’s a bit like when everyone freaked out when Apple decided to change the iPhone cable, everyone panicked for a few months, but now a couple of years on, no one cares, but everyone is glad they did.
I honestly think clinging onto the past while JavaScript rockets forward is a mistake, and I’m glad Meteor is supporting the wider ecosystem. So yes, I feel for your situation, it does suck. I’m in a similar boat with some Angular 1.X apps that we want to migrate to 2.X. Or old react apps before the split from React to React + ReactDOM. That’s life, though, all those systems have improved massively, even if they do hurt a few projects in the short term.
I’d say if you’re happy with 1.2, stick with 1.2 for those past projects. Move to 1.3 for new ones. In production, you ideally shouldn’t jump to update everything for every release, unless there is a major new feature you need.
That would be fine short term, but long term I am less worried about new features (I’m pretty satisfied with features), but more worried about any critical updates to vulnerabilities/security/etc.
@joshig the main problem is not the imports, it is the refactoring hurdles that come with it. Yes properly structuring the code beforehand and creating module based facades are two most basic things that should be implemented anyway. But, every app comes to a point (even within days/weeks) where names should be changed, files should be moved or split and javascript editor/ide offerings are far less than complete in terms of refactoring support. Even reportedly successful refactorings turn out to miss a few things, thus introducing subtle bugs or more overhead.
But yeah, this certainly is a move forward, but javascript’s (and Meteor’s) “move” does not always feel that forward. Or it does not feel well executed.
In fact, javascript/meteor move feels so sporadic and backwards-problematic that I strongly disagree with this statement:
In production, you ideally shouldn’t jump to update everything for every release, unless there is a major new feature you need.
Because you never know if two minor versions ahead, you’ll be left so behind that it would become impossible to even publish hotfixes due to any one of javascript core, meteor core and any library/package out there having abbandoned support for your deployed version. So you are kind of forced to keep up and even ahead.
Thank you for this excellent breakdown of the current state of refactoring in Webstorm. These are exactly the type of issues I was hoping to discuss in this thread.
Suffice it to say that Atom is basically in a similar state, if not even a bit further behind. Atom has a quick-move package, which doesn’t look at imports or file content at all. It has a refactor package which is a glorified version of ctrl-D multiselect, but doesn’t handle files. There’s no drag-and-drop. Files can’t be quickly divided. Not sure if moving a file would trigger a reload in the jsimport package, but the cynic in me guesses that it wont.
The bright side, however, is that with Atom we have the source code to all these packages (and more). So we know how to build up the imports map using jsimport. We know how to create the IDE menu selections using quick-move. We know how to search for import statements using refactor. Getting a working file-move refactor utility in Atom is doable by the community.
Excellent questions. The cynic and realist in me expects it will fall to the community.
Although I would like to state for the record, that there is a fairly large group of experienced devs in the Meteor early-adopter community, who view Meteor as the most sophisticated refactoring tool in the Node ecosystem. Meteor has historically attracted junior developers because it let them build Node apps without having to manage Node require statements; and moving files around and the build system watching and rebuilding automatically amounted to magic. However, that same build system has also attracted senior developers because they see it as an entire refactoring pipeline with filewatchers, file scanners, autoimports, file writing, etc. And that refactoring pipeline has to be rebuilt as of Meteor 1.3 to accommodate ES6 import statements.
So this is something that MDG has historically owned. Whether or not the choose to continue owning it… I got no clue. But this isn’t something new or some type of feature request. This is very much a core feature of the build system that’s been around since 0.3 or 0.4 days that’s possibly being removed, if the guide and statements about imports being the default in future releases are any indication.
[quote=“seba, post:11, topic:25176, full:true”]
Yeah, especially if you split up your app in multiple packages, it becomes very hard to move files around. And no editor knows what to do with ‘meteor/…’ imports. For packages you publish publicly it becomes almost impossible to refactor.[/quote]
Another great consideration. We’re definitely getting enough to create a bullet-point list of recommendations and an action-list. Will need to add meteor/.. imports to that list.
This should be a recommended best-practice in the guide and in example applications. Thank you for pointing this out. There’s a world of difference between absolute and relative paths in import statements; and I’m beginning to get to the point where I can’t see anything good about relative import paths.
# GOOD
api/ui/startup/utils
# BAD
./style
# REALLY BAD
../../../../../api/etc
I know for a fact that the Meteor Chef Base app is currently shipping using relative paths. That’s how ./style statements crept into my current app; and it’s slowly-but-steadily spiraled out of control into import-hell.
And looking outside the Node community, the C# and .Net community uses qualified namespaces, which are basically absolute paths.
So, can anybody think of a situation where relative paths are essential? Meteor packages? Npm packages? Otherwise, I think there’s a case to be made to update the Meteor Guide and various styleguides and best-practices to use absolute paths in import statements.
So, can anybody think of a situation where relative paths are essential?
I think we should move away from packages anyway, (since we are embracing imports) so that’s no longer an issue for the long run, but here are my pet peeves:
since meteor accepts imports from the imports folder only, and that’s not a top level project folder, all imports have to begin with /imports/ (or imports/ if it is to be considered a special imports root, per the build system (meteor) configurations)
at least webstorm gets confused with absolute paths. when I tried it, it completely lost the ability to resolve module exports properly. this leads me to think that absolute imports
absolute paths may mean different things depending on the environment (browser/node), you may even end up trying to read from the file system root!
it is also different from the perspective of the build system. for example, typescript and es6 treat paths differently (so I heard)
Anyway, if we actually do find out that using absolute paths is working fine, I’d still use a mix where files enclosed in a module use relative paths between each other and modules import from each other using absolute paths. Combining this with index.js files that rexport from the module’s private files, we’ll get a proper structure which we can later than have the ability distribute as public/private npm packages as well.
I second that. I’ve been watching this thread wondering why everyone has found it so difficult because I haven’t felt that way. One of the answers is that I used absolute paths for everything not very very closely related to the module I’m in. So, for example, if I’m in /imports/api/groups/methods.js and I need to import something from the users api, I use the absolute path, ‘/imports/api/users/users’. But, if I’m importing something else within the groups api, I use the relative path, ‘./groups’. This has made it easy to move whole groups of associated files around and just do a global find/replace on the path.
Of course, I started using modular languages 30 years ago and had no IDE support for at least the first 10 years - never really got used to the refactoring tools.
My perspective on the wisdom of introducing imports goes like this.
We’re moving to NPM as a packaging system, a move that will open doors both to the Meteor community to employ and deliver more packages to more users and to the NPM community to benefit from more of Meteor’s goodness.
NPM currently uses “require”.
We were fixing to have to start using “require” all over the place and encounter all of these problems anyway.
But ecmascript has already decided the winner in the war amongst systems for modularization and chosen “import”.
Everyone, including NPM, will eventually change to “import”.
Ben has given us a wonderful gift in that instead of moving to “require” and then having to shift again to “import”, we can jump straight to “import”. I applaud the timing of this! It had to happen and better now than later!
Furthermore, by implementing it as an NPM module, “reify”, he is bringing “import” to the whole NPM community, now. Note also that he was careful in his latest “reify” code to extend the ecmascript just a little, in a way that NPM absolutely requires, so that it can FULLY replace the “require” syntax. i.e. the latest versions allow import to be used in conditional and nested scopes. No more "require"s required.
All of this and other changes coming in 1.4 and 1.5 puts Meteor in the position of entering the NPM community on the bleeding edge instead of the trailing edge.
When at the bleeding edge, development tool pains always occur. We simply need to be as proactive in pushing the tools as MDG has been in pushing Meteor into a position where it can join the larger community and pull it instead of dragging it.
I am right this moment spending a portion of my time trying to get modifications introduced into at least one of the parsers that ESLint uses, either espree and/or babel-eslint, so that conditional and nested imports don’t cause parsing errors that kill all analysis of the modules containing them. i.e. I identified an upcoming problem that will effect not only our community but anyone in the NPM community using ESLint and Ben’s “reify” package to replace “require” with “import” and I’m working to fix that problem before people see it. I’m not an MDG employee, nor is anyone paying me for any effort I’m making today.
So, I applaud everyone on this thread exploring what WE can do to move development tools into position and hope that this discussion will result in solutions that can be put in the guide for everyone’s benefit instead of requests for someone else to do something. If you can’t do it yourself, you can still help greatly by searching for things that are open source and close to what is needed (as I’ve seen some are doing) or brainstorming positive ideas that might give inspiration to those on here who may have the skills and time necessary.
Well, actually I use it a bit more nuanced than I first said.
I do use relative paths for files I know will always reside in the same directory. Like index files and my scss and template imports of my angular components. And a disadvantage as @serkandurusoy also points out is that editors don’t get absolute urls. But that last problem might disappear if we completely move isopackages to NPM packages. But we’re probably still quite a bit away from that.
You are absolutely right in all of your assertions that the contents of the “module-name” in the imports syntax were not specified by the committee. Every build tool has the freedom to interpret it differently. I believe though that Meteor and NPM both will be using it as relative to the app or package directory. We just need to get our tools fixed to support our build system’s interpretation of “module-name” which I don’t believe is unusual.
Note that ESLint was at first marking absolute paths as unfound and a fix for that system is already available in the form of the NPM package “eslint-import-resolver-meteor”. I have submitted a PR to the Guide to add this.
Btw, does somebody know if the name of the index file is configurable?
I’d like to name it _index.js (so prefixed with underscore), that way it would always show up at the top of my directory in webstorm (and most other IDE’s).
