Issues we had moving our MongoDb to MongoDb Atlas from Compose.io

We’ve had a great experience. Idk what the difficulty with migrating is, you can do it with mongodump/mongorestore.

They offer more options, are cheaper, and have much much better tooling. Being able to query backups without downloading them is super cool.

Also, Compose has been having lots of downtime and host disconnects recently. Part of the reason we switched.

It’s a weird problem. My backend app can connect to the MongoDb cluster but the login to the backend app from my frontend app isn’t working. We’ve tried all sorts of things but this was never the problem with Compose.io - not sure if it’s a Atlas problem but it is the first time.

We can login when we run the backend app locally though. No whitelisting on Atlas before someone asks.

Why do you think I have opened two tickets with MongoDb? Because I don’t need help?

If you’re a customer of them then you should know they offer first 30 days of support for free.

Also, like stated we’re in beta still, not in production but close to launch if we wouldn’t waste our time with a sudden problem that never occurred on Compose.io.

I’m telling my experience, if that is bashing then maybe it’s because so far it’s anything than a perfect experience. Why do you think now even a VP from MongoDb has apologized? Because I bash them?

We all know that in order to win customer over from the competitors every entry barrier must be removed. Even a single question too much asked in your sign-on process can be a showstopper and leading to less new customer.

So if pointing out that MongoDb indeed can remove a lot of barriers by providing an easy way to win Meteor clients who run their database on Compose.io is bashing a company then I might misunderstand the meaning of bashing. To me bashing is unsubstantial, without proof.

I think I’ve highlighted that there are enough hurdles in the current process. Even when their help desk tried to help. It was my employee and not MongoDb help desk who figured out it was all caused through different MongoDb version that lead to us not being able to import the data in the first place.

As for the free instance, why not limit it after 30 days? Everyone can try Atlas for 30 days without the current restrictions of no oplog, only a max of 100 transaction per second (which makes it impossible to guess if the performance is better, same or worse).

There is actually a long list of features that aren’t working in the free version. I do get it that they don’t want customer to free ride a production environment. Everyone needs money to run their business but with my 30 days model suggested above it would give us a fair test ride and a like-for-like comparison with our current provider.

Another point, why is it so damn complicated to convert from the free “sandbox” version to a paid version? Another unnecessary hurdle build in by MongoDb. See here for details: Migrate Data from an M0 (Free Tier) Cluster to a Paid Cluster

Lastly, I do see that you’ve just joined Meteor forum and this is your first post. It might be that this is just a coincidence and you do indeed work as an Application Developer for North 40 Outfitters.

Well I hope it’s that way as it would be really lame for any MongoDb employee to not identify himself when commenting on here. Right?

So happy to have you on board at Meteor :wink:

Have a great day too!

Unfortunately that’s exactly what didn’t work for us because of the different MongoDb versions used

It actually is a coincidence. I just joined because I am checking out a project called Reaction. I also happen to be a Certified MongoDB Developer and when I saw this I felt like responding.

I understand you are having a bad time, but you don’t seem to understand that you are using a product outside of its intended scope. I understand that you wish the free tier was more than it is, but it isn’t.

On the other hand, as a certified MongoDB professional, I would have been more than happy to help you out. Instead, you post something titled “Don’t move your MongoDB to MongoDB Atlas!”, with an exclamation point. Then try to say you aren’t bashing it?

It seems like your biggest frustration is not being able to import data from an older version of MongoDB to a newer version though. If that’s so, it has nothing to do with Atlas and you would experience the same frustration just standing up your own 3.4.x version of MongoDB. Am I wrong there?

You might have set the mongo_url and oplog url incorrectly. Oplog url won’t
work for their free tier as mentioned above.

And I’d get in touch with their support. They’ll respond quickly. We just
went through this process and we were using the wrong connection strings.

Elie

Interesting, we migrated from 3.0 -> 3.4 without any issue

Yup. You really can’t use the free Atlas tier.

You should use Mongo 3.2 and Oplog. Neither available on Atlas free tier.

If you don’t want to pay $60 per month for their M10, go with mlab free version (and then migrate to Atlas when your db grows) or self host.

First rule of marketing, if you want attention (clicks) come up with the right headline

My biggest frustration is that it’s not working, free or not. See this message in my command line:

2017-06-12T20:14:00.637+0100 W NETWORK  [ReplicaSetMonitor-TaskExecutor-0] No primary detected for set Cluster0-shard-0
2017-06-12T20:14:00.637+0100 I NETWORK  [ReplicaSetMonitor-TaskExecutor-0] All nodes for set Cluster0-shard-0 are down. This has happened for 501 checks in a row.

Please see also this picture which I posted earlier:

So when you can’t connect to your MongoDb cluster via Compass, MongoBooster, Command Line and their web URL then you’re not disappointed?

I’m not a MongoDb developer, so I’m expecting more from a managed service like Atlas where the MongoDb devs are actually sitting on the other side of the screen. That’s why I have paid and will be willing to pay a premium for experts to ensure that my apps are up and running fine, 24x7

Please see above that I’m in touch with their support since days, hence the links to JIRA (which I understand now aren’t visible to you guys).

Yes, they respond quickly, something that MDG could learn from! Did I just bash another company? LOL

Yeah you did… well good luck with whatever you’re doing. You’ll get further with an attitude change.

1 Like

So I pointed out that I’ve been already in contact with their support and that I wrote it at the beginning of my post. You suggested that I should get in contact with them. Why does that require a change my attitude?

Hmh. Don’t think so. There are surely other points in my post were your suggestion is more adequate but not on that particular post.

Just some update with my ongoing discussion with MongoDb support (which hopefully might help others in the future):

They are actively working with MDG on solving the Oplog issue with their free M0 tier, see this:

As was previously noted, the issue at hand is likely related to the limitation imposed onto Atlas’ Free Tier cluster in which the oplog may not be tailed using the $regex operator. We understand that this limitation is problematic for Meteor users and have thus contacted some core members of the Meteor team to review Meteor’s oplog query patterns to ensure that we may provide the necessary support within Atlas’ Free Tier for Meteor developers. This collaborative effort is in-progress; we hope to have additional insights available by the end of this Summer.

They have also generously provided me with some free credits to try out their M10 tier and see if that solves the problems.

Lastly, they also agree that this is most likely a Meteor problem:

Knowing that you are able to connect to your Atlas cluster from your Laptop but not from Meteor does suggest the problem lies within Meteor.

This might be related to the bcrypt issues referenced in another thread here in the forum as I’m affected by these.

Our experience with Atlas has been fantastic. We use ObjectRocket by RackSpace and mLab too, so we’re in a good position to compare.

We are now moving all of our production systems to Atlas.

Exporting from ObjectRocket was as simple as running this command:

mongoexport --username [username] --password [password] --db [dbname] --host syd-mongos0.objectrocket.com:35023 --collection [collection] --out [out.json]

Notice that we’re using an encrypted connection to the ObjectRocket service. Your connection may differ.

We then import into Atlas with this rather long command:

mongoimport --host [name]-shard-0/[name]-shard-00-00-abcde.mongodb.net:27017,[name]-shard-00-01-abcde.mongodb.net:27017,[name]-shard-00-02-abcde.mongodb.net:27017 --ssl -u [username] -p ‘[password]’ --authenticationDatabase admin --db [dbname] --collection [collection] --drop --file [out.json]

This is also encrypted and also uses the replica set. We’re also adding the --drop flag to drop the target DB before importing.

Hopefully this will offer some help.

1 Like

I would like to give Atlas some credit, because it’s been very smooth so far (6 months of experience) and it’s been working great with Meteor. Reliable, great backup system, ok pricing and a good way to migrate from other providers (using mongodump). I’ve been using mLab before (5+ years), which is also a great provider, but we needed the encryption that Atlas offers (for free).

I don’t have much experience with the free tier, but the one I tried worked fine. Without oplog that is, but what do you expect from a free tier… And btw, you don’t need oplog for testing purposes. It will fallback to polling.

So moving your databases to Atlas seems like a safe bet.

Well, not in my experience. MongoDB as SaaS has been working fine for many years now.

That is very kind of them. Hope you return the favour and update this posts title… I think they deserve better.

1 Like

I’d like to give an update and indeed that put Atlas into a different light.

  1. Connect problem - turns out to be a firewall problem in certain locations I was using whilst on business travel

  2. Cluster down - it wasn’t down, we couldn’t connect to it. So it wasn’t an Atlas problem, it was a Meteor problem

  3. App down - explained in another thread here in the forum, we were using an older version (as we had bad experience in moving to the newest version too fast in the past) which didn’t had the latest npm driver for MongoDb 3.4.4 which Atlas uses. Hence Compose.io with their 3.0.x (?) version worked fine but it clearly wasn’t Atlas error. We moved to 1.4.3.2 which updates npm-mongo to 2.2.24 and our frontend app didn’t crash every minute anymore. Thanks to @abernix for providing a quick and correct solution here. The real Galaxy support was responding too late and with an useless comment (“Oh we see your app is running now and logs are gone so all is fine”).

In the beginning Atlas customer support was causing some of the problems described in this thread as their recommendations were going in the wrong direction but as soon as their VP took over (it was a different ballgame). They tried really hard and in comparison to other SaaS my company uses (Galaxy, Compose.io and Modulus) they are 2nd best now (still I rate Compose.io as the most responsive and best support.

Will edit the title accordingly

Just to add I moved from mLab to Atlas after trying out compose.io on the way. This was partly pushed by our need to run out of AWS Sydney but after trying all of these I’m very happy with Atlas. Their Compass product leaves a bit to be desired but the actual Atlas DB service I found easiest to use amongst from offerings. With Atlas, we run 3 projects, each on their own cluster, and have never had a problem.

We moved a bunch of our databases from Compose to Atlas in Feb. So far we’ve had a great experience. We still use compose for a lot of our data, and our planning to migrate over as soon as possible.
From a perspective of newbie in DB deployment etc (which we are), here are the reasons we choose to move to Atlas:

  • Newer version of mongodb available with SSL encryption, wired tiger (which not available on the Classic Plan)
  • Better pricing for our needs

However when it comes to exploring data and querying the db quickly, we still miss Compose’s online interface.

1 Like

It seems like the free tier of MongoDB Atlas has some limited support for accessing the oplog. Is this a recent change?

For example, it only supports queries on the oplog using string namespaces. Is this still enough for Meteor to work properly or does that require the full range of features?

EDIT: Meteor’s oplog does seem to work on the free Atlas M0 cluster. At least, I set up the MONGO_URL and MONGO_OPLOG_URL and haven’t gotten any errors yet.