High performance pattern using flow components

1

Hi all, we have been developing an app for some time, and have been using Flow Components developed by @arunoda et al which has been great.

Our app is for calendaring and displays a grid composed of a table which can have over 1000 cells. Each cell can be empty or display one of many different icons to indicate the status or that scheduled event. These icon states are derived from several data sources.

To process this table we may need to subscribe to hundreds up to near a 1000 records.

We have a date selector that allow us to move between months. The problem that we were encountering was extremely poor performance the more records that we were subscribed to.

We discovered the problem was too many queries to client side mongo, because the data for our calendar is derived from several different sources that we needed to check. Since client side queries do not have an index, these lookups were extremely cpu intensive and our application would hang, sometimes for 10-20 seconds.

The way that we solved this was to create a component cache. We placed this cache within an autorun of the component that makes one query through our data collection, and then parses the results and formats the data as we need it.

Each record of the cache is then stored as a key : object pair in the cache like { key1 : {object}, key2: {object}}. We actually created a multi part index using id’s and dates from our records like { _id1: { date1: {cacheObject1}, date2: {cacheObject2}}, _id2: { date1: {cacheObject3}, date2: {cacheObject4}}}

The result has been nothing short of miraculous for our performance. Our render times now are near instant, with no percievable pause.

This pattern could be implemented in other ways besides using flow components, but flow components worked very well for our purpose.

I’d be interested to hear other ideas/options when dealing with a large number of records, that can’t be simply iterated through in the template.

1 Like
2

I’ve done a very similar optimization in a very similar online reservation project, building a snapshot of the calendar data and querying only for live updates where necessary had increased rendering performance from tens of seconds to milliseconds.

It was an over the weekend thing as a gift to a friend’s new business so I’m sure there’s much room for improvement, though.

Also, I had to depend on mizzao:timesync to mitigate time discrepancies and found out that the diffault value of 1 second updateTime is a killer for rendering performance if that data is used to reactively update the calendar grid, so I’ve increased that resolution up to around 1 minute.

3

That sounds like a good, pragmatic solution :smile:.

There’s also a new package which adds indexes to minimongo. I’ve yet to try it, but it also looks like an interesting approach to solving a similar use case.