So today a very weird error occurred to me for the first time.
Following steps happened prior to the error:
- Due to a very cpu-intense operation of importing and matching 10.000 images into my osrtio:files collection, I did exactly that on my local machine. So I first removed all items from my local
images
collection, imported all the files and uploaded them to my AWS S3. - I exported my
images
collection viamongodump
toimages.bson
. - I copied the bson file to my production server and
mongorestore
'd the file - without--drop
of course. - I ran my “image matcher” on my production server, which matched my 10.000 images to several items in another collection. … now the weird stuff started:
- after the matching had ended, the CPU on the server stayed at 100% for to the
node
process for about 1-2 minutes. no idea why. - from now, the whole app started to behave very weird: at some point the connection to the MongoDB seemed to completely crash, the
node
process spiked at 100% several times, no client could receive data from mongo anymore. Some client threw errors looking like this one. - I tried:
-) restarting both my docker containers.
-) restarting mymongod
process.
Both did not fix my problem, after several seconds or maybe one minute the whole thing “crashed” again.
What fixed it in the end, was mongorestore --drop
-ing my items
and my images
collection, so the images
was missing my additional 10.000 images, and about 10.000 items of my items
collection were without a imageUrl
attribute again - that’s all.
I just don’t understand … all so so weird. I feel like it may has something todo with indices or some mismatch behind the scenes of what ostrio:files package expects to be in the images
collection or something like that.
Fun-fact: Neither the node processes, nor the mongod process threw ANY errors. Nothing.
Any ideas on this?
best, Patrick