Suddenly can't upload images to newer DigitalOcean droplets

I have been using jalik:ufs and jalik:ufs-gridfs packages for uploading images. So far there hasn’t been any problems, but recently I suddenly can’t upload images on new DigitalOcean droplets anymore.

The old droplets are still doing fine, so I also tried creating new droplets and deploy old projects to test. The result is the same, old droplets are fine, new droplets unable to upload any images.

They are all Ubuntu 16.04.1 x64 droplets. The latest one which can still upload images was created 2 months ago.

My projects are all at Meteor, and I am using mupx to deploy.

The server log shows below message after upload progress is at 100%:
Exception while invoking method ‘ufsComplete’ MongoError: Can’t get executor for query { files_id: “EWi4MJY3WMhywSS3o”, n: { $gte: 0 } }
at Object.Future.wait (/bundle/bundle/programs/server/node_modules/fibers/future.js:449:15)
at [object Object].ufsComplete (packages/jalik:ufs/ufs-methods.js:66:20)
at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1711:12)
at packages/ddp-server/livedata_server.js:711:19
at [object Object]..extend.withValue (packages/meteor/dynamics_nodejs.js:56:1)
at packages/ddp-server/livedata_server.js:709:40
at [object Object].
.extend.withValue (packages/meteor/dynamics_nodejs.js:56:1)
at packages/ddp-server/livedata_server.js:707:46
at tryCallTwo (/bundle/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/promise/lib/core.js:45:5)
at doResolve (/bundle/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/promise/lib/core.js:200:13)

at Object.toError (/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongodb/utils.js:114:11)
at /bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongodb/db.js:1196:31
at /bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongodb/db.js:1905:9
at Server.Base._callHandler (/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
at /bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
at [object Object].MongoReply.parseBody (/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
at [object Object].<anonymous> (/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongodb/connection/server.js:446:20)
at [object Object].emit (events.js:95:17)
at [object Object].<anonymous> (/bundle/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:207:13)
at [object Object].emit (events.js:98:17)

I have also tried other packages like cfs:filesystem and cfs:gridfs, also work on older droplets only but not newly created ones.

All projects are able to upload images locally, only fails on production on new droplets.

I am not sure how much more information I need to provide. Any help would be greatly appreciated.

It’s not good practice to upload static files to instances, did you consider using amazon s3 etc? Obviously does not answer your question, but it’s not sustainable over time. D.O. May have changed their local temp storage path or permissions also, if you wanted to check with them.

I think I will consider amazon s3.
Wasn’t an option before because the rates for amazon is just too confusing for me. Guess now I have no choice but to try.

Could you elaborate on why it’s not sustainable over time to upload static files to instances?

I’m guessing because of the storage restrictions on DO, but that’s somewhat different now as you can add more storage (in certain regions anyway).

However, it’s a +1 for Amazon S3 from me as well. With so many npm packages available for S3 integration (such as this one) it just makes sense to use S3 in my opinion!

I see. Thank you for the explanation and the link.

Mainly because they are meant to be stateless, no files, data, or settings are stored on an actual instance. They are loaded from a central filestore, database or from environment variables etc. This allows you to scale many instances as needed, or shutdown instances and move users around without affecting their application state. If you store files on an instance, and move to another, you would lose those files. It may work out ok on a small personal app and there’s nothing stopping you from doing it, it just causes issues as you grow.

1 Like

Thank you so much for your explanation. I never considered the dangers of storing data and files on DO instances until now, thank you.

Those are great points actually, thanks for sharing!

I have a similiar problem using Wekan and a MongoDB. I run both of them in Docker containers.

When I try to upload a file attachement through Wekan a MongoError occurs, and the Wekan Container crashes:

SA GridFS - ERROR! { [MongoError: Can't get executor for query { files_id: ObjectId('586e33bd3183220600b6fe5b'), n: { $gte: 0 } }]
  name: 'MongoError',
  ok: 0,
  errmsg: 'Can\'t get executor for query { files_id: ObjectId(\'586e33bd3183220600b6fe5b\'), n: { $gte: 0 } }',
  code: 17241,
  codeName: 'Location17241' }

        throw message;
Error: Error storing uploaded file to TempStore: Can't get executor for query { files_id: ObjectId('586e33bd3183220600b6
fe5b'), n: { $gte: 0 } }
    at EventEmitter.<anonymous> (packages/cfs_collection/packages/cfs_collection.js:161:1)
    at EventEmitter.emit (events.js:117:20)
    at Stream.<anonymous> (packages/cfs_tempstore/packages/cfs_tempstore.js:343:1)
    at Stream.emit (events.js:117:20)
    at Stream._error (/built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/gridfs-stream/lib/writestream.js:190:8)
    at /built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/gridfs-stream/lib/writestream.js:128:28
    at /built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:524:58
    at /built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:474:20
    at /built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/db.js:1194:16
    at /built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/db.js:1903:9
    at Server.Base._callHandler (/built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
    at /built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
    at [object Object].MongoReply.parseBody (/built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
    at [object Object].<anonymous> (/built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/server.js:446:20)
    at [object Object].emit (events.js:95:17)
    at [object Object].<anonymous> (/built_app/programs/server/npm/node_modules/meteor/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:207:13)

This problem first started around the some time as yours: at 2016-12-23. Maybe these two errors have the same cause?


this problem is because of MongoDB version 3.4.1 that was released 2016-12-20.

Downgrade to MongoDB 3.4.0, this keeps your existing data, and uploading attachments works with it:

sudo apt-get remove mongodb-org*
sudo apt install -y mongodb-org=3.4.0 mongodb-org-server=3.4.0 mongodb-org-shell=3.4.0 mongodb-org-mongos=3.4.0 mongodb-org-tools=3.4.0

I’ll try to figure out later what has changed in MongoDB 3.4.1.

In Wefork it’s this issue:

I did not realize that there was a more active fork for wekan available. How did you find the fork? The wekan network page refuses to display anything useful (“Too many forks to display”)

Thank you for the clear up. I was having problems deploying Meteor 1.4 apps with my Windows machine using mup, so have been using 1.3 with mupx. Recently I bought an iMac and had no problem deploying 1.4 with mup, and somehow this problem also went away.

I am very new to web development, so right now I’m just assuming having a Mac will make life easier somehow, despite how much I don’t like this machine.

I have merged Wefor/Wekan fork back to Wekan:

I am now maintainer of Wekan.

At Wekan we still use newest MongoDB v3.2.x because of this bug, it’s now v3.2.13.

Meanwhile Wekan has been upgraded to Meteor v1.4.x and Node v4.8.1. That node version is used because newer ones cause 100% CPU usage.