Optimizing Meteor App Efficiency (My app is slowww)

help
1

Bear with me… this is my first try at developing a Meteor app, and I am learning Javascript/HTML/CSS as I go!

So… what do I try to do? Build the most complicated web app a noob could try to make, of course!!

My app gathers market data for around 900 cryptocurrencies via one API and publishes statistics, information and graphs with the data. It is a full fledged social network with forums, and almost everything is likable and followable. People can create portfolios and track their values, and get placed on leaderboards for the best performing portfolios. And also submit links and technologies, and rate or discuss them.

As you can tell, there’s a lot going on there. It took me three months of many hours per day to develop the prototype which is many thousands of lines of code. You can see it here: https://www.decentralized.tech

It is being hosted on Meteor Galaxy’s hosting service on a double standard server on a single container. I have the APM activated too.

The app ran fine when it was only me testing it. It was little sluggish at times, but everything always worked.

I incentivized some people to come try the app yesterday and everything broke. I was only seeing 10 connections or less, but publications were taking forever to publish, methods were taking forever too, and sometimes not responding at all, and things were not even loading at times… such as modals with reactive data.

I have two ideas as to how to increase performance:

  1. I do my subscriptions via my Iron router config, and with each route change it kills the subscription from the previous route and subscribes to a new subscription for the current route. I remember reading somewhere that there is a package that will keep these subscriptions alive inbetween route changes. This ought to save some server resources by not resubscribing to data that was already subscribed to.

  2. I noticed on the large datasets that the oplog observer is really slowing things down a lot. Some subscriptions dont need to be reactive, so I need to disable the observeschanges feature on certain subscriptions. That should save some resources too!!

Can anyone give me some other ideas as to how to speed things up, or some good articles to read? Thanks for your time and help!!!

2

Here are a few screenshots from my APM: https://imgur.com/gallery/ysooU

3

If you click in the chart on the APM, you should be able to view a trace. From there you will be able to see a breakdown of what operations are eating up time

4

Yeah, but to be honest, most of what is in the APM is gobbledygook to me, and there’s so many statistics that it’s hard to know what to look for! Even if I see something that is slow, it is hard to discern how to make it faster!

This is a good read for anyone that is interested: https://medium.freecodecamp.com/scaling-meteor-a-year-on-26ee37588e4b

It helped me know of a great package that can help: mixmax:smart-disconnect
https://atmospherejs.com/mixmax/smart-disconnect

Now I am trying to find the package that stops iron router from closing the previous subscription on every route change. EDIT: Found it here. It doesn’t look too hard to implement. https://atmospherejs.com/meteorhacks/subs-manager

5

Just an observation;

at https://www.decentralized.tech/market-data Cryptos collection has 50 documents.
navigate to https://www.decentralized.tech/charts/bitcoin Cryptos collection has 923 documents after a few seconds.
navigate back to https://www.decentralized.tech/market-data and it wil render a panel for 923 Cryptos.

I think Cryptos alone will mass it up on both client and server. First thing to do is page/limit the publications or if you really need to load so many documents at once on the client consider non-reactive.

6

Noticed:

  • Too Many Template Errors (Exception in template helper: TypeError: Cannot read property ‘toFixed’ of undefined … )
  • On each page publish what you need.
  • For Graphs, aggregate the data on server then send it to client (Use Meteor Methods)
  • Consider using projection on MongoDB Query, for example you send field priceArray24H from Cryptos-document. Do you really needed it?. Minimizing documents size will increase the performance.

For debugging you can use meteor-devtools plugin in chrome (It works also on your production website).

Good luck! and don’t give up :slight_smile:

7

“navigate back to https://www.decentralized.tech/market-data and it wil render a panel for 923 Cryptos.”
Thanks, I was unaware of that. It was an issue that popped up when I introduced kadirahq:subs-manager upthread. I created a different SubsManager for the cryptocurrency selection boxes:

I do page/limit the Cryptos collection where I can. The only place I show most of the document fields of the Cryptos collection is on the /market-data page, and there is a selection box to select how many documents to return. The default is 50, which is why you got 50 documents when there are actually 920+ documents.

When you go to the Charts or a few other pages, I have a Cryptocurrency selection box which I use the Cryptos collection to aggregate data in. On those pages I only publish on the server and find on the client the two applicable fields to that collection: name and a unique id for each which is used for routing purposes.

This “cryptosNamesAndIds” pub/sub slowness is puzzling to me. I have both fields indexed (name and id).

2560×1080 94.5 KB

2560×1080 132 KB

2560×1080 126 KB

Obviously, the sub downloads the information quickly, but a lot of time is spent setting up the OverserveChanges reactivity of the pub/sub. However, these names and ids almost never change, and new ones are rarely inserted into the collection. I guess I need to figure out how I can disable observeChanges on this collection??

8

ObserveChanges is running for 900+ documents. Can you share your pub/sub code ?

I didn’t get the idea. Are you intending to make it SEO friendly? I hope you are not creating 900+ routes in advance.

How do you aggregate ? With mongodb aggregation or your code?

9

Do you have OpLog tailing enabled? If you don’t, that’s for SURE what your problem is.

1 Like
10

Pub:

Meteor.publish('cryptosNamesIds', function (query, options) {

    this.unblock();

    if (options) {

        check(options, {
            sort: Match.Optional(Object),
            limit: Match.Optional(Number),
            fields: Match.Optional(Object)
        });

    } else {

        var options = {};
    }

    options.fields = {
        name: 1,
        id: 1
    };

    return Cryptos.find(query, options);
    this.ready();
});

Sub:

NamesIds = new SubsManager();
NamesIds.subscribe('cryptosNamesIds', {}, {
                fields: {
                    'name': 1,
                    'id': 1
                }
            })

Yes, I hope to rank decently by making it SEO friendly. Poke around and see how I have my routes set up. I have wayyyy more than 900 routes. Probably in the high 1000s. I thought this was good though. The most visited routes will automatically get pushed up in rankings, and the more obscure routes will rank high for their obscure key words.

I have admittedly been having trouble getting SEO setup though. I have the dferber:prerender and mdg:seo packages added, since I am hosting on galaxy. I tried with just the mdg:seo package, but after waiting a week or so Google still thought my site was empty. So, I added dferber:prerender along side it. It is working better, but still not very good.

I was using lookback:seo to add meta tags, but I don’t think it was working. I have never been able to view the web site source and see the meta tags there even with the plugin and adding ?escaped_fragment= at the end of the URL. I switched to kadira:dochead today and I hope that works better. I am just adding a description meta tag and changing the page title dynamically for each route change for now.

I am aggregating with my code. I haven’t looked into mongodb aggregation yet really. The site has been running much faster since I’ve limited the amount of documents that are queried for by using “infinite scroll” techniques. My wait time is much better the past week than is for the past 30 days on both pub/subs and methods.

11

Thanks for the reply. OpLog tailing was not enabled at first, but I have since added it as per the meteor galaxy APM suggestions. I have received quite a lot of efficiency boosts after enabling that with an oplog mongodb user and also limiting my queries to smaller subsets of documents. But some queries I cannot limit, such as the cryptosNamesIds publication, which needs to return all cryptocurrencies in the database.

I don’t think my app is that slow anymore. I have made a lot of changes, which seem to have things runnign better. I’m not sure it will scale, but it is at least loading much faster than it was. I am more concerned about SEO at the moment, because I am having a hard time getting it to work. SEO in Meteor/web apps is much harder than any other web development I’ve done in the past, namely sites from scratch in html/css or wordpress.

I am using the following packages: kadira:dochead, mdg:seo, and dferber:prerender … but not sure if they are really working. Searching my website on google by “site:www.decentralized.tech” makes it seem like something is not working.

12

You need prerender.io, but if you’re hosting on Galaxy you have a free account up to 250 pages. Sadly, it caches even bot requests. You can probably white-list certain pages only.

Your SEO is not correct, no. It’s a pain, good luck.

13

Thanks. I will try using a plugin like that to fix my SEO. That is a good idea… I can’t believe I didn’t think of that!

I read that the keyword meta tag hasn’t been used by search engines in years though, so I probably won’t fool with that.

Seeing as though I have a lot more than 250 routes, it seems like I will need to switch from mdg:seo to paying for the prerender service.

Thanks for the insight!!

14

Yeah I would say you’ll need to make the jump.

You can check out my page, www.StarCommanderOnline.com with any of the usual SEO inspectors, and you’ll see it running properly. Know that it’s using Meteor, and it works.

I spent maybe… 2-3 days getting it working. Really complicated, I wouldn’t wanna do it again.

15

I solved the prerender issue. For anyone that’s interested:

I ended up removing both the mdg:seo and the dferber:prerender packages. Then:

meteor npm install --save prerender
meteor add ecmascript

Somewhere in server side code:
let prerender = require(‘prerender-node’).set(‘prerenderToken’, ‘key’);
WebApp.connectHandlers.use(prerender);

Replace “key” with your prerender token. :wink:

2 Likes
16

Looks like a gross fix, but I’m glad it’s working for you!

1 Like