Why is Meteor with Houston Admin using so much bandwidth & CPU?

packages
1

( Cross posted here: http://stackoverflow.com/questions/34844705/why-is-meteor-using-so-much-bandwidth-is-it-houston-admin-subscriptions/ )

Here is a strange scaling issue with Meteor. UNPUBLISHED collections seem to be affecting server usage. EDIT: This seems to be related to Houston Admin package.

REPRODUCE:
Here is the reproducible code: https://github.com/keyscores/houston_admin_performance_issue

SCENARIO A:
I started with a prototype app, which was aggregating/reducing 5 docs in ‘db.originalData’, into 1 doc in ‘db.reducedResult’. The only collection published is the tiny ‘reducedResult’.
All this looks very efficient:

  • 1% CPU usage
  • 20mb ram allocated

SCENARIO B:
However, when I test real data with 15,000 records in originalData, not published:

  • 100% CPU is used
  • 100% ram is used (1GB)
  • Memory swapping to Disk
  • 100% Bandwidth is used 10mb/S

This is strange because only the small collection * 1 record * is published. The large collection, originalCollection, is unpublished.

Seems like Houston Admin is using lots of CPU when connected to remote MongoDB instances.

Houston seems to create subscriptions on all collections, even those which might be intentionally hidden to the client (not published). It also seems occur on users which are not logged in. See the CPU timeline attached.

The problem is solved by removing Houston. The error also disappears when using a local mongo instance.

Issue ticketed here: https://github.com/keyscores/houston_admin_performance_issue

Infrastructure:
The mongo server is on a separate server (compose.io) than the Meteor front end (Digital Ocean via MUP)

1 Like
2

Because CMS + Meteor is not a good idea until you get rid of reactivity.

1 Like
3

Meteor is great for CMSs even with reactivity. I dont use Houston because i like to build my own, but i think the high load is a complete different issue here. Houston does for example some mongo pulls of all collections every 10 seconds, even without any data changing or anybody connected to the app. With a big dataset that can be quite some traffic. 10000 records might be ~15MB every 10s and it works through those records every 10s which explains high cpu loads.

4

I agree, there are many other interesting problems to solve that Meteor is better suited for.