Okay, so I have this desired architecture for loading data with offline support. Here are the basic requirements, in no particular order:
- I don’t need full reactivity (actually, in this case I specifically don’t want it, since I’ll be doing pagination), and I want something which scales (uses less memory on the server) - and methods are faster anyway (in my experience - WAY faster). Any one of these requirements may not be enough to avoid pub/sub, but I think added up they make a compelling case. So I’ll be delivering the bulk of the data over meteor methods instead of using pub/sub.
- I’ll use Ground:DB on the client side, so I can still write nice queries. All views will be fed from Ground:DB instances - never directly from methods or subscriptions.
- I need to be able to render my React tree on the server, so on the server we’ll need to pull data directly from Mongo Collections, instead of Ground:DB. I figure some kind of sleight of hand should work for this - feed Mongo on the server, and Ground on the client.
- Since we are querying the Mongo sources for data during SSR, it would be a shame to query them again during hydration on the client - so I’m also looking for a way to capture all those queries, serialize it on the server, and hydrate the data on the client from there (like the old
fast-render package does for Blaze).
- Once all this is working, I’ll probably use pub/sub sparingly for some realtime aspects - like subscribing to comments on active displayed content. Even there I’ll probably do something cheeky, like subscribe to one field, and when it’s updated, use a method to grab the full data (to avoid holding full documents in memory on the server). This may be a premature optimization though - I might just use DDP. Not sure yet.
For most of that I already have a solution - yay! But I don’t have a great way to do data hydration - is there a fast-render like package for React based apps?
One other challenging thing is that for SSR, I need to know what data to wait for (and Ground:DB requires an additional wait step to make sure it’s stores are ready - I found that out the hard way on a previous project). With pub/sub it’s actually relatively easy to make that work by polyfilling the subscribe method (which normally doesn’t exist and throws an error on the server). For pub/sub, we can just capture all the subscription requests on the server, pass those parameters to the client, and do all the subscribing and wait for that to finish, before finishing up subscribe. But I’m not sure how to do that on the client side - but I have a rough idea.
I think I’ll probably do something similar to the subscribe model, but write my own method/actions. Since I’m pulling data from a Mongo Collection on both sides (ground:db on the client) I can leave that code alone. All I need then is an API which contains enough data to populate the Ground:DB collections from a method - and record that on the server to pass to the client, do all the method invocations (or hydrate that data from a fast-render like data source), and then continue on rendering! I think that’ll work. I won’t be able to invoke this action in my tracker wrappers (withTracker) because it’ll run too often - instead I’ll have to write a Provider component, and do it on mount. This will actually be great - I can then use a Consumer to provide the mongo collections - Mongo on server, and Ground on client! I think this will work!
(Sometimes it helps to pretend I’m talking to someone to work these things out.) Wish me luck!
1 Like
Sounds like an interesting approach and mix of tools. Keep us posted! 
Very interesting approach. I hope you send some updates on this topic as it evolves.
https://atmospherejs.com/staringatlights/fast-render supports SSR with React but I haven’t used it.
I got the first bit of it working - using regular collection on the server, and ground on the client. It’s a bit sloppy though, because if you include ground in the server bundle, it complains (even if you don’t use it). So I went with using dynamic-imports and an if statement to branch between them. But that’s weird, took workarounds to get it to work within the tracker, and requires an extra step between the tracker and the Provider. I could use a different tracker solution, maybe one which only tracks a single method, and use that single stateful component for the provider, but I think it would actually be better to write two different providers, one for the server, and one for the client. I think the only way to do that is to create a package. Then I can use the same import statement on both client and server, but it’ll import the right component on each. I’m not aware of any way to do that without a package, though maybe there’s a babel transform for that? I’ll probably just use a package.
Here’s what I got so far:
import { Meteor } from 'meteor/meteor'
import React, { Component } from 'react'
import { withTracker } from 'meteor/react-meteor-data'
import { ReactiveVar } from 'meteor/reactive-var'
import { Methods } from '/imports/utils/methods'
// We can use a closure for this, since it's a once per app-run bit of state.
const CollectionVar = new ReactiveVar(null)
const withContent = withTracker((props) => {
const limit = props.limit || 10
const collection = CollectionVar.get()
if (collection === null) {
// avoid trying to import more than once
CollectionVar.set(false)
if (Meteor.isServer) {
import('../collections/content/Content').then((module) => {
CollectionVar.set(module.default)
}, (err) => console.error(err))
} else {
import('../collections/content/OfflineContent').then((module) => {
CollectionVar.set(module.default)
}, (err) => console.error(err))
}
}
if (!collection) {
return {
value: [],
loading: true,
collection: null
}
}
const content = collection.find({}, {
limit,
sort: { createdAt: -1 }
}).fetch()
return {
value: content,
loading: false,
collection
}
})
const { Provider, Consumer } = contextize('content', withContent)
class ContextProvider extends Component {
async componentDidMount () {
if (!Meteor.isServer) {
const { collection } = this.props
const content = await Methods.getContent()
// OfflineContent.clear()
for (let tile of content) {
collection.remove(tile._id)
collection.insert(tile)
}
}
}
render () {
const {value, children, Context} = this.props
return <Context.Provider value={value}>
{children}
</Context.Provider>
}
}
function contextize (name, connector) {
const Context = React.createContext(name)
return {
Provider: connector((props) => (
(props.loading || !props.collection)
? <div>Loading ...</div>
: <ContextProvider
Context={Context}
collection={props.collection}
value={props.value}>
{props.children}
</ContextProvider>
)),
Consumer: Context.Consumer
}
}
export {
withContent,
Provider as ContentProvider,
Consumer as ContentConsumer
}
Do get rid of the dynamic imports, I ended up creating a local package, with a small API - a single function to create a Collection (regular on server, and Ground:DB on client) and return it. I needed to use a package because I wanted a single addressable location to import from, and I don’t know of a way to do that in the standard folders. So once I have my either/or collection in the standard location, the rest is easy - and no more dynamic loads and “loading” during rendering needed!
The next step will be to figure out how to record queries, and dump out all the data they return. It works the way it is now, but it hits the mongo database twice with the same queries for every page load (one for SSR, and then again during hydration from the method).
1 Like
You’re doing some very interesting work! I hope you’ll consider making a boilerplate when you’ve got everything sorted.
To be honest, this is turning out to be a lot of work. I probably should have just used Apollo. 