I noticed that in the past months the quality of your work … was not the top priority of the team.
Today I notified Marketing @jessa and @fredmaiaarantes on direct message here about the recent social post about Classcraft which is sharing a 404 page of that website.
Also, recently a core security package, Passwordless, was published with bugs and pretty bad code quality not covered by tests.
Today I see an update to node-stubs NPM and the repo has no updated changelog. I understand what was done but please maintain the quality and details that we are all used to.
We’re also a little concerned of late. I know it’s been a difficult time with a lot of changes in personnel but some releases feel rushed, especially the accounts updates - although I do appreciate the transparency you’ve shown with dealing with them.
We’re currently blocked from updating past 2.5.8 as we feel there’s possibly some serious issues in the oplog converter. See these issues in general but more specifically this one. There’s very little input from Meteor devs there. Shouldn’t the stability of this core element of Meteor be the highest priority?
I also noticed the broken link and asked Jessa to fix it yesterday. Thanks for the private warning.
Unfortunately, the passwordless package was released with this bug already, almost a year ago. We certainly need to improve test coverage and general testing, and we’d love to get more help with beta testing.
Do you have any suggestions for us to get more help testing before official releases? We usually get very little feedback, and testing different scenarios is crucial to improving Meteor quality. We’ve been trying (very recently) to wait longer in beta, but I’m not sure if that’s enough.
About meteor-node-stubs, it was just an update to fix the GitHub URL on the NPM page. There was no new code.
Hey @marklynch, thanks for your understanding. It looks like the MongoDB oplog problem isn’t new, and we started (yesterday) by checking to see if it got worse in recent versions.
This doesn’t affect everyone as we haven’t noticed any downgrades on our apps and most customers have no complaints. So we need to be able to reproduce the problem to find the cause. This is @grubba’s current task, and he will likely reach out on the created issue very soon.
Our team got behind on joining tests with the changes related to MongoDB 5.0. We have multiple apps to check and it was a rather big task. I believe a lot of teams got behind with that particular release.
Then the security issues related to account started which made us less aggressive to update (i.e. we wait before updating). So we are behind on recent versions and therefore not able to join tests on the upcoming versions.
As I posted before, we need to have the confidence that new versions are up to the quality we were used to. And that will be back in due time once we are getting solid releases again.
That is one of the problems. It is still an update. Make it simple for developers to decide if this is a needed update or not. It does not matter if a period is added on a doc. The update will still show up somewhere e.g. version lens in vscode. And those who care will want to read what was changed. And the ones who cared are the ones not being given attention by not giving an update on the changelog
Do you have any suggestions for us to get more help testing before official releases? We usually get very little feedback, and testing different scenarios is crucial to improving Meteor quality. We’ve been trying (very recently) to wait longer in beta, but I’m not sure if that’s enough.
I see potential by using larger projects like Wekan or Rocket.Chat and create forks (or get them on-board) and create some CI-automation (like GitHub actions) that gets triggered when a new Beta Release is available and let them run all their full-app tests with this new Beta release. I think by having 4-5 of such applications with different packages, architecture etc. would be a good start. We also have some apps that could benefit from this. I just don’t know how to trigger the CI automatically
Another thing that I found in general is, that the tinytest infrastructure, of which many core packages rely on has never introduced a decent way for code coverage analysis. However, the majority of the package tests use it and it’s not known, how many lines / branches / statements were left uncovered.
I know that code-coverage alone is not an indicator of quality but I think it’s an important tool to uncover potential pitfalls.
In addition to apps, it might be good to run tests for some non-core Meteor packages with beta releases. Node.js does something similar with npm packages, and has it at least partially automated with CI: GitHub - nodejs/citgm: Canary in the Gold Mine.
Haha. I think the even easier “fix” is just to avoid using subscriptions as much as possible unless absolutely necessary (which I definitely think is a Meteor best practice at scale, even though it’s sad to avoid the reactive magic)
One more idea came to my mind on this topic! We should move forward to implement usage and error statistics in development mode (opt-in of course!) I think this makes it also easier for the core devs to prioritize and reduced the time and complexity on error reproductions.
Today I notified Marketing @jessa and @fredmaiaarantes on direct message here about the recent social post about Classcraft which is sharing a 404 page of that website.
