Meteor Data graphing app with HTTP REST system. Need help

help
21

All right, I have read up on the collection2 and simpleSchema. I like the idea of using the schema so that on any insert it creates a time created at. I was wondering does

`new Date()`

create a time with month,day,year, hour,second? example code I have for the schema. Or would I need to format the incoming Date This is all untested because I will have to redo a lot of code and just working on some small things.

`var station = new SimpleSchema({
    name: {
    type: String,
    label: "Station's Name",
    },
    longitude: {
	type: Number
    },
    latitude: {
	type: Number
    },
    createdAt: {
	type: Date,
	autoValue: function(){
		if(this.isInsert){
			return new Date();
		}else if(this.isUpsert){
			return {$setOnInsert: new Date()};
		}else {
			this.unset();
		}
	}
    },
    humidity: {
	type: Number
    },// Each sensor would look like this after.
});`

I really like the idea of only sending the newest 20 points from the server by a publisher. Is the publisher a constant hook? Is the publisher always sending the newest 20 points so that any new data will instantly be sent to the client subscription? If so I didnā€™t know it did that.

I do wish to show total number of data points per station. Would I just make another publisher and subscriber?

Also with template formatting would this be fine instead of having a Stations and station that I have a NodePage. When you click on the sidebar link to the Node(station) you want to visit it takes you to the NodePage and on there is

`<div class="Graphs">
    {{> windDirection}}
    {{> windSpeed}}
    {{> temperature}}
    {{> humidity}}
    {{> pressure}}
    {{> dewPoint}}
</div>`

Each of these is a link to a template for each graph on the page. Would this be fine or will this interfere with any onRendered orautorun? Each graph at the moment renders just fine and with spline seems to work great with just loading in the random data I have set for the testing.

You can see the full client side here.
clientsidegitHub

Theory question. Can a collection be too large for Meteor to handle? Because the theory is that there could be thousands of stations on this collection. And each station having around 45K points each. Also when I remove a said point from the collection is it like a delete row in excel where all the subsequent stations move up the ladder?

22

Hey @robfallows would it be all right if I put a field at the bottom of the main page with an honorable mention of your help with this project?

23

Creates a new date object set to the current time. See Date - JavaScript | MDN

Yes.

You could do that, but be careful - having an up-to-date count would likely end up being very CPU heavy on the server, especially given the update frequency youā€™re getting. Iā€™d be inclined to either do without the counts or have a count property on the weather station document, so what I originally suggested would look like:

{
  _id: 'OSU Node 1',
  gps: '134.234,78.545',
  count: 123456
}

and update that value using MongoDBā€™s $inc operator each time a new data point is added to the readings collection. If you ever delete readings you will have to reflect changes here, so itā€™s less easy than doing nothing :wink:, but it will dramatically reduce server load compared with counting documents.

Thatā€™s fine - itā€™s your app :smile:!

There are many MongoDB collections with millions or even billions of documents. As with any database, you need to ensure your design is sound: good indexes will be essential in a large collection. As far as Meteor is concerned, by getting MongoDB to do the heavy lifting via good queries on a good structure, we are able to leverage what MongoDB is designed for and provide Meteor with good quality, highly specific results.

Well, I guess you could think of it like that. If you remove a point from the database which is being rendered on a graph, reactivity will cause the graph to be redrawn, so you will see that. If the point is outside the graph ā€œspaceā€ you wonā€™t see anything.

Haha - Iā€™m flattered - but itā€™s not necessary. Instead, how about ā€œpassing it forwardā€. Take the opportunity to help someone with something - doesnā€™t have to be Meteor. Youā€™ll know when the timeā€™s right!

2 Likes
24

I donā€™t know why I enjoy this so much but anyway I ran into a couple snags. I was able to pull unique names out of the collection to show the names on the sidebar so users could click on a station and be taken to that stations graph page.

`{{#each nodes}}
    <li>
        <a href="{{pathFor 'NodePage'}}">
            {{this}}
        </a>
    </li>
 {{/each}}`

Then on ht javaScript side

`Template.sidebar.helpers({
    nodes: function(){
    var wow = _.uniq(_.pluck(Nodes.find().fetch(),"name"));
    return wow;
    }
});`

BTW I didnt know Meteor came with underscore js so helpful. Although for some reason the routes no longer work taking me to NodePage.

`Router.route('/:name', {
     name: 'NodePage',
     data: function() { 
         return Nodes.findOne({name: this.params.name}); 
     }
 });`

It worked when it had

`return Nodes.find();`

Since this new switch happened I have wanted to make a new collection with just the unique names of the collection with all the readings. Is there a way in simpleSchema to autamatically insert into another collection via the name section.

This one is for Nodes

`name: { 
     type: String,
     unique: false,
     max: 20,
     "????": function(){
	     if(this.isInsert){
                 Names.insert(this.name???);
	     }
      }`

While the Names Schema would have a unique equal true instead. Or is there a better way to do this with just using the Nodes collection that I am missing in the router. I just see some issues with trying to read data from one collection being referenced from another collection.

25

Iā€™m not sure what youā€™re saying. I cloned your repo and ran with Meteor and navigation works fine for me. The route is not defined like either of those above:

Router.route('/:_id', {
  name: 'NodePage',
    data: function() { return Nodes.findOne(
        // this refers to the currently matched
        //route.this.params access parts of route
        this.params._id);
    }
});
26

Iā€™m sorry I had not updated the repository. Just did so and now the code is up to date. I have some commented code out in my server and lib files because I was trying to add another collection but was getting an error so I commented it out so you could run and see the names on the sidebar no longer link to nodePage with their name as a /ā€œnameā€

`(STDERR) MongoError: E11000 duplicate key error index: meteor.names.$c2_name  dup key: { : "OSU Node 1" }`

this is the error when adding the new collection. I think I just did something wrong some where it was late and didnt fix it.

27

The reason the links donā€™t work is because the Nodes documents do not contain name.

Router.route('/:name', {
  name: 'NodePage',
    data: function() {
      console.log(Nodes.find().fetch()); // added this to see what was in Nodes
      return Nodes.findOne({name: this.params.name});
    }
});
28

All right. I added a waitOn in the router. A subscription if block around my template each block. I feel like Iā€™m missing a fundamental aspect of routing somewhere. I have been working on this all day and have read over many router fixes and subscription and publishing docs. The only place I can think there is an issue is in my Template helper.

`var wow = _.uniq(_.pluck(Nodes.find().fetch(),"name"));
 return wow;`

Where this will show every name in the whole collection not unique names. But the routes work.

`return Nodes.find();`
29

I fixed it. I guess I didnā€™t know that the router had to be passed an object to use the

`this.params.name`

I was only passing an array of names and not the object that held the name.

1 Like
30

Thatā€™s great! I started to look at this on Friday, but never having used IR, I had to start with getting an understanding of that!

31

As far as your Names schema and collection is concerned, you are definitely on the right track. I uncommented your Names simple schema and collection2 code and used this in the Nodes schema:

name: { 
  type: String,
  unique: false,
  max: 20,
  autoValue: function() {
    if (this.isInsert) {
      try {
       Names.insert({name: this.value});
      } catch (e) {
      }
    }
  }

Where name is a unique constraint in the names collection. However, if you intend holding more than just the name in this collection it may be better to move this out of the Nodes schema and into the main code body.

32

Thank you sir haha. Sorry it has been a while since I have been back on here. But things have been going well. I now have my data coming in and graphing with time perfectly fine. Also I have gotten the google maps gps mapping to work great as well. Although I have run into a bit of a snag.
@robfallows These are my last 2 problems with the site.

Like I said earlier I need to delete old points after 7500 points have come in. I was thinking of just looping through unique named collection items and deleting the first point if an insert comes in and there are more than 7500 points. The code I have should work. Although I put it into the server and nothing happened. Should this code be on the client side?

Also the reason I am asking this is that inserts will be coming in from an arduino with a 3G/GPRS antenna sending data via HTTP/HTTPs to this website. Would inserts catch POST requests from this Arduino? Or would I need to write specific code to break the incoming data into an insert format? I canā€™t seem to find any examples of an external device POSTing to a Meteor website and inserting into a collection.

33

Also I am getting this very annoying error
Uncaught TypeError: Cannot read property ā€˜dataā€™ of undefined
but I cannot seem to find out what is going on. It is on lines 82,84 on both my graph js files. windDirection.js and dewPoint.js

Graphs