I noticed some packages used directly from atmosphere stopped working after update to meteor 1.3, and authors of less popular ones just didn’t update them.
Using each package and just counting it should work - consumes development time, when some error occurs. I think it would be better to have 1/4 of available packages if i just could be sure, they work with latest version of meteor.
Best way is to write tests and run them automatically on each package in atmosphere. Package authors would have to add test to each package, and if it would lack tests - should be marked as untested. I would decide to pick less functional, but tested package in that case.
This gets back to both the latest-version vs stable-version debate, and the cathedral vs bazaar debate. Much easier to ensure that packages work together if there’s a stable release built using a cathedral model. Unfortunately, the hosting business model, which is how MDG is monetizing, has business incentives to follow a bazaar model and keep up with the latest trends; so it’s always going to be a moving target.
What you’re describing is a commercial product unto itself. We’ve thought about this a lot with the Clinical Track; and are considering forking Stratosphere and creating an integrated continuous/integration/QA server that does basically what you’re describing. There’s definitely people thinking along the same lines as you. But everybody will second-guess the design and architecture, unless it fits their specific business model. So the only practical way to get such a specialized QA tested package server seems to be to have an industry vertical that is mandated to have QA tested software. So… healthcare, finance, transportation, are the likely suspects.
We’re not too far away from having such a setup. We’re already pulling in all the package server data and cross-referencing it with continuous-integration build status on the Clinical Track release pages.