Hi, i did an app to show at least 180.000 rows of MySQL each one with multiple joins (at least 3)
Like (this is just an example, not my real DB structure):
Person (180.000 rows)
Family (at least 3 rows for each People)
Members (at least 40 rows for each Family at least)
SomethingElse (… at least 5 rows)
In the end this is information that i want to filter to obtain some Coordinates so by some events on the client i will fetch data.
Doing some JOINS takes seconds but using GraphQL & Sequelize… takes a little more (at least 4 seconds more). Im planning to use DataLoader but i have no idea how to implement it, i know there is an example using with graphql but im not sure how to use it.
This sounds like an interesting experiment, but I find it hard to provide any advice without an answer to the following three questions:
How much data are you actually fetching to display (surely not 180,000 rows)
Do you have all the necessary indices defined on your DB?
Do you have a working version in GraphQL without dataloader?
With the correct indexes, I don’t think the JOIN query in SQL should take a few seconds, unless you’re selecting a very large subset of the 180,000 rows.
1.- im actually fetching around 40.000 rows (yes is a lot but are coordinates points).
2.- Yes, i refined my DB to perfom the best i could.
3.- Yep, my GraphQL is running without dataloader.
My main thought, maybe is SEQUELIZE the problem, that is making a SELECT then a SUB SELECT then a SUB SUB SELECT and so on… am i wrong ?
Ok, so 40’000 rows is really quite a lot. Are you doing this to run some benchmark test, or is it an actual app you’re working on?
I’d be surprised if you didn’t have problems rendering all that data on the client at once. Or maybe you don’t need to render it all at once, and could make queries for just the content you need for that one screen?
In any case, it’s not that easy to use dataloader with sequelize, because it doesn’t let you control the queries that are made to the database. You could write some dataloaders by hand, and use sequelize or knex inside of those, but just for sending the actual query, not for using their models.
Believe it or not, rendering 40.000 lat/lng data array takes 9ms…
Yes i think Sequelize is not the best way to perform queries… I will follow that path, doing my queries by hand and using Seq or Knex only to send it. Or maybe its me knowing a few of it lol.
I started using hand made queries. It improved significantly (80%) my site loading time. Sequelize is awesome but handles in a weird way the things. Maybe i lack of the technical knowledge to use it.
