Best data structure for IOT application -> dynamic collections

Hello,

I am writing a simple HomeIOT application that my local ESP8266 devices with DHT22 sensors will send their data to every few minutes. It will have a dashboard (currently in materializecss) that allows uses to add devices with a name, passcode and the fields for the data being sent. I have easily made the devices and dashboard however I’m currently a little stuck and needing assistance for the layout.

I was planning on utilising mongodb to store the data using a simple rest API. This would allow each device to post to a url which would then verify the name, the code and then push the fields into a new document that would be rendered in the charts.

However, I am unsure how to dynamically add a collection when a device is made, such that each device has its own collection for the data being sent.

Should i dynamically make a collection that can have data added to it… i.e.:

Device name: ‘Inside_DHT22’ … then i was considering making a collection something similar to: ‘data_Inside_DHT22’ which i would then call in the rest API.

I was hoping to make similar collections for every device made, as to avoid mixing data for multiple devices in a single collection.

Is there any advice you can give me on the best way to structure this? or would one collection suffice with each document containing a name field to identify the device it is from?

If you were to suggest making a collection for each device ( assuming they push new data every 5 minutes, and runs 24/7), and if so, how would is the best way to do that when the number of devices being added is unknown?

Regards,
Jeremy

I’m curious as to why you think data should not be mixed, given that it can be reliably identified?

It’s generally good practise to keep like data together - and it (hugely) simplifies the design and workflow.

The most important thing (however you do it) is to ensure you set up indexes appropriately.

Im unsure to be honest, I’m new to Meteor and mongo and just preferred having it separated, but having posted this i have combined the data :slight_smile: How would you set up the indexes? any particular advice for a new person to Meteor?

You should determine the indexes from your query requirements (MongoDB is not really any different from SQL in this regard). So, MongoDB gives you an index “for free” on _id, but it sounds like you will at least need an index on device as well - but you will likely have more. For example, if you want to return documents in the order they were written, you may need to add a createdAt field (you could look at aldeed:simple-schema with aldeed:collection2 or matb33:collection-hooks for this). It may then make sense to use a composite index ({device:1, createdAt:1}) rather than multiple, separate indexes. Again, only you understand your requirements, so you will need to do this analysis yourself.

To add indexes to a collection in Meteor, the normal practise is to put the code in the server’s Meteor.startup(), for example:

Meteor.startup(function() {
  MyCollection._ensureIndex({device:1, createdAt:1});
});
1 Like