Big product catalog. Is Meteor good choice?


#1

Hello everyone!

I need to build quite interactive product catalog (for web and mobile), around 200000-300000 products with combinations and a few images for each product.
All data come from XML so I need to read it and to save to Mongo and after work with this data on front-end using diffrent filters to select products and variations.

My concerns are:

  1. is it easy to make each product page searchible by google with Meteor?
  2. is codebase for mobile and web different?
  3. is Meteor right tool in general for that or may be it would be better do it more traditional way?

What do you think, shall I try?

Thank you for your comments!


#2

Yes it is. As long as you set things up correctly, search engines are not a problem. Set up a page cache server, and you are good.

If by mobile you mean mobile website, no, you can serve both from the same Meteor instance. If you mean a mobile app and a web server, although the setup will be a little different, it will still be the same Meteor instance serving the mobile apps.

I have experience building such a system with traditional methods (PHP+MySQL to be exact), and I assure you, you won’t lose anything by building it with Meteor.

There are many things to be careful about when building such a system, but these two are the most influencial things you need to pay special attention to:

  1. Setup a cache server, and make sure to cache your homepage, product and category pages.
  2. Optimize your database and indexes very carefully.

Also, SEO best practices in a traditional system and a Meteor system does not really change as much as people would have you believe. Your HTML will (should) still look the same.


#3

As far as SEO is concerned, the canonical Meteor way (if Galaxy is an indicator) is to use prerender.io (either self-hosted, or as a paid for service).


#4

Hey @miksdigital, I agree what @necmettin has said. There won’t be any issue with Meteor. You can consider using https://reactioncommerce.com/ rather than developing from scratch. You’ll need to disable some functionalities of Reaction Commerce which are not needed by you.


#5

Hey @deligence1 !

Thanks for sharing!
I like your link and I’ll give it a try!


#6

Frankly, modeling categories in mongoDB is overly complicated. A big catalogue, you should stick with a relational database, meaning meteor may not be the best fit.


#7

@jitterbop , thanks for your reply!

I am not very experienced with e-commerce in general but my data comes from XML and I consider convert XML to json like that (source) :

{
  sku: "00e8da9b",
  type: "Audio Album",
  title: "A Love Supreme",
  description: "by John Coltrane",
  asin: "B0000A118M",

  shipping: {
    weight: 6,
    dimensions: {
      width: 10,
      height: 10,
      depth: 1
    },
  },

  pricing: {
    list: 1200,
    retail: 1100,
    savings: 100,
    pct_savings: 8
  },

  details: {
    title: "A Love Supreme [Original Recording Reissued]",
    artist: "John Coltrane",
    genre: [ "Jazz", "General" ],
        ...
    tracks: [
      "A Love Supreme Part I: Acknowledgement",
      "A Love Supreme Part II - Resolution",
      "A Love Supreme, Part III: Pursuance",
      "A Love Supreme, Part IV-Psalm"
    ],
  },
}

At least I can see the whole picture of a product and can manipulate it easily.
So my task is to get data from XML, convert to json and work with it on front-end with a lot of interaction from user.
That’s why I look at meteor as a proper tool.


#8

I think as long as your data design is sound and you understand how to work with MongoDB you should be fine. However, if you want SQL you can use MySQL and PostgreSQL today - it won’t be quite as invisible as MongoDB, but you can get reactive subscriptions, for example.


#9

@robfallows , I’d like to use as much native meteor tools as possible. I hope this way will be easier than trying to link different technology between each other for no reason.

After converting XML input to json I will have this data type in backend and can work with this data on front-end.
So my data and data flow should be “meteor-way”.

Am I right?


#10

My bad, I overlooked you are starting with xml.


#11

Yes, XML.

So for my noobs opinion it is a good sign to start thinking of meteor technology :slightly_smiling:


#12

Meteor is not prescriptive about how you store/use your data. At the moment MongoDB is the best fit, but Apollo is going to blow the whole data layer wide open. You should use MongoDB if it’s the right choice for your project, but don’t infer that “the Meteor Way” means you must use MongoDB.

Neither the seven principles of Meteor (anyone remember those? They seem to have vanished from the docs):

  • Data on the Wire. Don’t send HTML over the network. Send data and let the client decide how to render it.
  • One Language. Write both the client and the server parts of your interface in JavaScript.
  • Database Everywhere. Use the same transparent API to access your database from the client or the server.
  • Latency Compensation. On the client, use prefetching and model simulation to make it look like you have a zero-latency connection to the database.
  • Full Stack Reactivity. Make realtime the default. All layers, from database to template, should make an event-driven interface available.
  • Embrace the Ecosystem. Meteor is open source and integrates, rather than replaces, existing open source tools and frameworks.
  • Simplicity Equals Productivity. The best way to make something seem simple is to have it actually be simple. Accomplish this through clean, classically beautiful APIs.

nor the current layer diagram:

mention MongoDB.


#13

For Categories you can use https://docs.mongodb.org/ecosystem/use-cases/category-hierarchy/

I’ve used something similar but more adapted to my circumstances. Works well.


#14

Thanks Gentlemen!

Your replies encourage me to start.

Especially @keithnicholas , @deligence1 , @robfallows and @necmettin !


#15

#16

maybe not necessary to group/nest these. Why not keep it simple:

  id: "nmZqmzYW5b5q8LvtJ",
  sku: "00e8da9b",
  type: "Audio Album",
  title: "A Love Supreme",
  description: "by John Coltrane",
  asin: "B0000A118M",
  weightGrams: 6000,
  widthMM: 10,
  heightMM: 10,
  depthMM: 1
  listPriceJPY: 1200,
  retailPriceJPY: 1100,
  },

as for

savings: 100,
 pct_savings: 8

who is driving who? one could just be computed based on the other . . .


#17

@maxhodges , Max , thanks for your comment!
It was just an example of data.

In reality I need to work with XML feeds from different suppliers so data structure will be also different for each product.
It’s like price aggregation tool.

I like this presentation. Slide #41 and later explains why I want to try MongoDB in my case.


#18

Brief tangent; we’re going to need some product data for our app, do manufacturers tend to make that info available nowadays? (I realise my question is grossly unspecific!)


#19

is it easy to make each product page searchible by google with Meteor?

It’s very easy to make things searchable in meteor, as opposed to other frameworks

is codebase for mobile and web different?

Slightly, but not really

is Meteor right tool in general for that or may be it would be better do it more traditional way?

I would say you would be better off using knex with postgres and socket.io with an express web server. You can create a socket stream that will publish to the client as it finds results. Here is a tiny example of what I mean

That’s not to say that you can’t use meteor, it’s just that this really isn’t taking advantage of anything meteor really thrives at.


#20

Yes, there are plenty of XML feeds (files) from different suppliers for dealership and affiliate programs.
Do you need some?