Scenario: We have an application that is currently using 1.3.2.4 that is coming close to production release. We do NOT want to update to 1.4 yet as that takes some verification and we want to postpone that until we get our heads above the water. Our application has a number of meteor packages we wrote that we use to structure the subsystems of the application. We have unit tests for these. These unit tests used to run without issue. All of the sudden, without any code changes, they fail. We cannot even check out older versions of our code and run tests that we know succeeded. When we run our app, it uses bcrypt 0.8.5. When I run test-packages on one of our packages, it decides to use ~/.meteor/packages/npm-bcrypt/.0.8.7_1 and fails with a dyld: lazy symbol binding failed: Symbol not found: _node_module_register.
If we didn’t change anything on our side, why did things start failing with this error? If previous versions of our software whose tests used to work fail now, why is this?
I’ve lost a lot of confidence in Meteor over this.
More information on this: The problem is rooted in the use of accounts:password. The application and the packages use version 1.1.8 of accounts:password (the version is explicitly specified in the packages). The application runs no problem; the unit tests fail on the packages with the above bcrypt error.
I got a similar problem. After updating to meteor 1.4.* a critical issue occured with css-modules (Another long story, more details here: https://github.com/nathantreid/meteor-css-modules/issues/53). so i was forced to downgrade Meteor to the last workable version 1.3.2.4. But after downgrading, another issue occured with accounts-password package. When i run application it’s crushing with error W20160826-21:50:55.069(3)? (STDERR) dyld: lazy symbol binding failed: Symbol not found: _node_module_register.
I tried to change node and npm versions to the ones used by meteor in ~/.meteor/packages/meteor-tool/1.3.2_4/mt-os.osx.x86_64/dev_bundle/bin. tried to rebuild with meteor npm rebuild, reinstall meteor… but all efforts were in vain.
And now i don’t even know which of problems is worse. When i was not to able to use postcss or generally impossibility to start application.
I think the error is due to a recent update to Meteor core. Afaik this comes from Meteor no longer pinning individual packages to a Meteor version number. The upside is that individual packages can be upgraded without waiting for a new meteor release. The downside is the problem you’re encountering now.
This change has been made recently, not sure with which version though (can be found in the logs). Perhaps that’s why and as rob suggested pinning the versions file to a specific version might work.
Yes, but again, the application works fine because it is getting the right version of bcrypt. It’s the unit testing of the packages that is erring out. I’ve tried locking the version of bcrypt in the packages as well to no avail.
I get that being flexible with package versions and not pinning them to a particular release is great but it shouldn’t affect existing versions. If we don’t change anything in our product and the product breaks (in our case, our package unit tests), that is completely unacceptable. It should be possible to check out a version of our application that was created four months ago that was originally on meteor 1.x and it should run on meteor 1.x exactly as it did previously. Otherwise, if a regression occurs, how do we bisect to find where it was introduced? The application’s behavior is affected by updates we don’t want to incorporate.
Not only can we not do this, our package unit tests broke within one development sprint - things were working at the beginning then part way through things broke. Meteor is trying to be too cute with its versioning and it is failing. Unless a user requests explicitly that they be upgraded/updated, it should not happen, period. There should not be an update then a mimicking of previous releases, it should simply not happen.
I don’t want to sound overly negative here but this is pretty bad.
If that is the case I think here is your answer, if not I hope this info is useful for others:
There is a comment below that post I made which might make it more understandable. If your use-case is internal packages, so not published to Atmosphere:
In short:
Public atmosphere packages
Public packages on Atmosphere need to work in many configurations. They should be flexible with version numbers so they can work for example in both Meteor 1.3 and 1.4. Off course that’s not always the case. So what happens, and that is the strange part here, is that when running:
meteor test-packages
you get different versions compared to your normal environment in which you develop. You get the lowest versions possible. Where you likely use the “latest” versions in your app.
The problem
That’s how things go wrong. This difference is not shown clearly, you just see that it’s building. You also don’t see this in for example your versions file, because Meteor uses a separate build for this.
The fix
In most cases you can fix this specific issue with adding:
api.use('accounts-password@1.3.0');
into your package.js file. That will force the build process to take the most recent version of accounts and as a result also the most recent version of crypt.
How to unit-test well
That being said, this is not nice for people who use packages as internal organization in an app. Since you want to run your unit tests against the same version as your app. Otherwise you have no idea what is going on. I am not sure how you can enforce that.
TinyTest support
Addition here should be that tiny test is not a supported solution, in the guide you can find the supported ones: https://guide.meteor.com/testing.html But since I know there are quite a lot of people who have stable sets of tinyTest tests it’s for sure important to have this stable.
Thanks for the detailed response. However, I am relatively familiar with the rules around versioning of packages vs applications (packages use oldest matching version, applications using latest). This isn’t the issue here though - remember, the application is correctly using bcrypt 0.8.5 whereas the package that uses accounts:password (@1.1.8) while doing a test-packages is trying to use bcrypt 0.8.7_1 which is the opposite of what one would think it should be doing.
Also, while we are using tiny test, this isn’t the issue AFAIK. The target package can’t even be loaded by the test driver so using another test driver won’t change that.
Our packages all use specific versions when calling api.use() as well, so that’s not the issue, either. We are explicit with .versionsFrom() and we also use versions in our api.use(). None of these changed.
There seems to be an implicit recommendation that we use the latest version of packages. That only makes sense for the current, latest version of the application. When we bisect to find where regressions occur, we want the exact versions. Otherwise, there’s no point in having a revision control system - it only manages part of the history.
Regarding the comment:
Since you want to run your unit tests against the same version as your app. Otherwise you have no idea what is going on. I am not sure how you can enforce that.
The way to enforce it is to not have meteor automagically update itself unless the user requests it and honor what the package is asking for. If a user is using meteor at version X, no auto-magic stuff should occur to change that. I don’t want the latest packages to be implicitly selected. This makes no sense.
I understand the need for critical security level fixes which would allow a package to be updated without the application’s control but that should be strictly controlled and should be proven to work. There needs to be a better separation between urgent package fixes (like a security fix) vs “we should just choose the latest package that’s available, because you know, packages”. I’m on board with urgent fixes being automatically selectable but just selecting the latest that’s available by default for no good reason, nope.
But again, it still doesn’t make sense that bcrypt 0.8.7.7_1 was selected from a package unless accounts:password is being too cavalier with its version selection (this is my guess).
Was not aware of your level of knowledge on versions and as it’s quite complex I hope my post will help more people who encounter this kind of issues. Let it be clear I am no expert on the constraint solver, just using Meteor a lot.
We also use TinyTest as well and it works great, it just is strange on versioning because they are not fixed as in the linked github issue was shown. That can create strange situations.
Now on topic:
I was looking for 1.1.8 for accounts-password and I think it got introduced here:
The bcrypt issues have appeared after that. Did you try to run with accounts-password on 1.3.0 and does that solve your issue? Or is that not possible from your current Meteor release version? Because at that time the bcrypt in accounts password was at:
Where it’s now at:
Now that can be connected to the nodeJS upgrade we had but also there are related issues to bcyrpt which have been solved, including a workaround by @benjamn I think which introduces a JS implementation. Not saying it’s the best way it it may help you to proceed with your application.
Maybe you can find some more details on that on the github.
Internal package versions
I have seen more issues with explicit version numbers because the constraint solver wants to get things in it’s own way. In the end, for releases, your versions are fixed because they are in the versions file. But if you want off course you can also fix them in all packages like Meteor does.
We work without that which makes upgrading much easier and we test each update separately. That way the constraint solves finds the best solution. And then off course we fix them when all fine.
Also there is the issue of reproducibility:
When we bisect to find where regressions occur, we want the exact versions. Otherwise, there’s no point in having a revision control system - it only manages part of the history.
You can do that. But only on the app itself. test-packages will behave differently. You commit your versions file and that fixes all versions in your app. So, if we reproduce, we can test the full app and all end-to-end tests. But: The test-packages mode doesn’t have a versions file. It will try to find what it thinks is best.
I don’t want the latest packages to be implicitly selected. This makes no sense.
Agree, and it can work out if you hardcode all dependencies all the way to the bottom. But: if there is an external package for example which doesn’t: The magic will come in in test-packages mode. There is, as far as I know, not much you can do about it in that scenario.
An option could be to build, if possible without minimizing, and also storing the build output as your real history. That’s about the only way to have a real hard copy.
More issues
There are many more issues around this, I suspect your read some, but this one might be informative:
But again, it still doesn’t make sense that bcrypt 0.8.7.7_1 was selected from a package unless accounts:password is being too cavalier with its version selection (this is my guess).
Agree, it’s strange that the solver finds a higher version while in testing mode compared to production. On the other way it’s the mix of all packages which the solver finds. So yes it can end up with a specific version being higher in the end result because the total result is better as far as I get the principles.
But it should, as @dgreensp mentioned, be fixed to specific versions. Since the move to NPM I don’t expect much changes on this end by Meteor. You may like NPM modules because that is less magic in dealing with versions. accounts-password doesn’t have an explicit version, just a minimum, so it will have that bit of magic in it.