External API call - limit frequency

help
1

I populate a collection via server-side external API call. My API provider offered lower subscription fees if I limit the number of calls, but I’m not sure how to define a frequency.

I would ideally like to make the call at certain times of the day. For example every 6 hours. But is Meteor.setInterval(apiCall, 21600000) really feasible? What if want to pick non-standard intervals? Like 8am, 1pm, 4pm, 8pm 12am. Mutliple setIntervals that find the pattern?

I’m pretty new to Meteor and development in general. Through some searching and re-searching I built the call as shown below. After getting a little more used to it all, I’m realizing I probably don’t need to do this via publish, as I’m not subscribing to ‘getCollectionData’ anywhere, and the inserts/updates in the try block get the data into the collection. Should I just give this function a name and use the setInterval and find a good pattern (for example 6 hours / 21600000ms)?

Meteor.publish('getCollectionData', function(){ 
  try {
    var response = HTTP.get("apisite");
        _.each(response.data, function() {
            for (var i = 0; i <= response.data.length - 1; i++) 
              {
                var exists = Collection.findOne( {ID: response.data[i].ID} );
                if (!exists) {               
                  Collection.insert(response.data[i]);                  
                } else {
                  Collection.update({ID: response.data[i].ID});
                }
              }
        });
   } catch(error) {
        console.log(error);
   }
});

One last question - it seems right now this call runs every time the server restarts (which is ALOT during development…). Is this because it’s in a “publish” statement in a server side js file?

Any thoughts/suggestions greatly appreciated!

2

You could use a sufficiently big interval as you suggest, but a better way (and multi-server safe) is to use percolate:synced-cron.

1 Like
3

Thanks Rob! I didn’t think to search “cron” at atmosphere. My thinking was limited to “frequency”…

1 Like