Redux + DDP , no minimongo

Showcase
22

Lol, whoever was in the leaderboard app, I was in there too messing around.

23

@lai that’s awesome, can you provide the URL to your fork when it’s ready?
Structure the files however makes the most sense. I’m just getting started with 1.3 so I don’t yet know how to appropriately setup the file structure and manage exports/imports.

24

Yea, I will let you know. I started out fresh and used your server code. Right now everything is working beautifully. Next thing is optimistic updates. Last night I’ve written most of the code to handle Redux optimistic updates using Meteor’s livedata mechanism, however I haven’t gotten a chance to test it because it was 1am LOL. So tonight’s the night!

2 Likes
25

In the meantime we can discuss what would be best way to slice Store.
For me it seems something like Collections, ReactiveDirs, ReactiveVars, Custom.
Any other suggestions?

Also from DDP side, we should not forget that there can be more DDP connections active,

If I remember correctly, having 2 apps using same collection name it was not possible to create both on client, as they must be global objects.

But when we will be in full controll at DDP endpoint, we should be able to handle both of them by including some DDP indetifier probably already in the registerStore call, as there is already collection name used as string identifier.

Or are we attaching after that and cant handle name conflicts @lai ?

26

I have not handled the case of an external DDP connection, but I will definitely look into that. If my guess is right (edit: I was right!), that connection object that is returned from DDP.connect should also have a registerStore call on it. So my abstraction function could probably take in an optional DDP connection object, as well as an optional alias name for the collection name in case two ddp connections share an identical collection name.

27

What do you mean by this? The case where two DDP servers ( presumably Meteor apps running on a server ) use the same collection name, And a client wants to connect to both servers and use both collections ?

28

Yes, as there is no namespacing suggested in documentation I believe there will be some conflicting names.
And it should be easier to count with it from start when designing API than introducing breaking change later on.

1 Like
29

Hey guys finally here it is. DDP collection data synced straight to your Redux store!

Unfortunately, I couldn’t get optimistic updates to work due to some implementation details, you can see more details about it in the repo. However, if you get the chance to see and understand what I did, perhaps you might come up with an idea of how to make optimistic updates become doable, it does involve getting into the guts of how livedata works though.

2 Likes
30

Minimongo has two main purposes in Meteor today:

  1. A global cache that keeps your UI consistent and allows you to temporarily update it for Optimistic UI purposes. It has built-in tracking so that you can roll back the optimistic updates when the server has responded. This is the most valuable part.
  2. A simulation of the MongoDB API to make it easy to write code that does optimistic updates and the real updates with the same syntax, and to avoid learning a new query language for reading from your client-side cache.

Would be pretty cool if you could do just (1) with Redux! (2) is going to become less important when your server isn’t running MongoDB.

4 Likes
31

I almost got the optimistic UI to work but got stumped on some details regarding how it would be used. See this issue for details.

32

@lai I haven’t looked into this project specifically for optimistic UI but my first draft for using it looked like this (note though my example is ajax but the req/resp concept should be the same):

  • UPDATE_POST action triggers REST/GraphQL call to server and return the post data, note a uuid is generated and used for the ID here

  • Reducer inserts post into collection and saves old post data to a prevUpdateState key.

  • On success mine ignores any further action but an UPDATE_POST_SUCCESS could clear old state

  • On failure an UPDATE_POST_FAILED is dispatched with the ID of the doc. The old doc state is replaced with _.revertDoc(state.posts, id, oldData) Other reducers can act on this action to throw down a dropdown alert and revert back to the edit page (which auto prefills forms with oldData).

Inserts/deletes are similar but failure actions will dispatch the ID so that you can pluck it out of the array (a simple underscore mixin helper makes removing by ID a one liner).

Does this make any sense? (not sure if i’m leaving out bits)

I tend to keep things simple unless I have to do otherwise. I’ve found the more you complect things the more bugs you run into. However, with a bit of convention, making some middleware to do this automatically would be possible.

Would you (or anyone else) want to hack on a simple proof of concept app? I’m hoping to find time at the airport this weekend to spike out something and throw it on GH… it would be beneficial to everyone (including me) to hash out any issues i’ve unknowingly incurred.

33

btw, 4days old redux-optimistic-ui https://github.com/mattkrick/redux-optimistic-ui

34

That looks cool, however, I wanted to use Meteor’s internals to implement the optimistic UI.

35

What you described is similar to what Meteor’s livedata already does under the hood and that’s what I’m trying to leverage because it’s not minimongo-specific. That internal Meteor API is how I was able to get the subscribed data directly into the Redux store and not minimongo. What you’re proposing seems to deviate from DDP?

36

So core should be implementing saveOriginals() and retrieveOriginals() .
For me after looking on minimongo implementation and merging it with mine low redux knowledge, it seems kinda simple
saveOriginals() cloning state subtree representing related collection to _savedOriginals
retrieveOriginals() diffing current state subtree representing related collection VS _savedOriginals and generating id - object pair

retrieveOriginals returns an
object whose keys are the ids of the documents that were affected since the
call to saveOriginals() and the values are equal to the document’s contents at the time of saveOriginals.

I am quite lost in looking up relevant parts of beginUpdate/endUpdate to identify where and how exactly is complete lifecycle wired.
I feel like if we keep the results of subtree diffing in retrieveOriginals before transforming for DDP id/doc pairs, than during decision logic we should be able to easily revert. But that would break tracking changes in actions.

And DDP deal with stuff per collection. But actions can theoretically affect any number of collections, so we need to track actions and parse these into collections and than again that id/object pairs for DDP tracking per object. And then revert some actions if requested in case of server method fail.

Or just keep diffing and in case of revert request fire REVERT action where we merge all these diffs we kept from retrieveOriginals per collection .It does not sound so horrible after all.

37

https://github.com/looshi/redux-ddp (redux-ddp)

  • depends on a generic node ddp.js npm and thus can be meteor agnostic?
  • this paves the way for a generic ddp-redux implementation that can be used in both generic client applications without meteor, and in meteor client applications without meteor-ddp and tracker?
  • meteor-ddp, tracker, minimongo could maybe be removed from the meteor client?

https://github.com/rclai/redux-ddp (meteor-redux-ddp?)

  • depends on Meteor.connection and the meteor-ddp package(s) and thus is meteor centric?
  • what would be the benefits of tight meteor integration over a generic npm redux-ddp client solution?
  • tracker, minimongo could maybe be removed from the client?

redux-ddp(+.map, .reduce, .filter) sounds like it covers to some extent the same bases as relay-graphql, with the added benefits that with meteor-ddp and tracker removed from the client, it will still allow authentication, server method calls and use of the meteor accounts packages when mongo is used on the server?

38

The original idea was just an exercise to get the DDP data straight into the Redux store and see where that takes us: tracker (and consequently reactive-dict and all host of reactive_x that depend on tracker) and minimongo can be successfully taken out as a result. Also, I was doing this in Meteor because @looshi, the OP, was doing that in his initial example code. So a Meteor-agnostic approach wasn’t even on my mind (not that I don’t value it).

While it is tightly married to Meteor’s livedata package, it is possible MDG is going to start breaking their core packages into NPM packages, which they claimed they were eventually going to do (according to some forum post). If that happens, then we can depend on a MDG-maintained DDP client NPM package instead of all the other ones out there.

@shock, there’s a bunch of stuff that happens between saveOriginals and fetchOriginals when it comes to how optimistic UI works in Meteor. If you look at the minimongo source, you’ll notice a _saveOriginal method, which saves the document state prior to the database write. The diffing happens in livedata in the process_updates.

So if I’m not mistaken the life-cycle is:

  • You call a Meteor method
  • The client-side method runs
  • livedata calls saveOriginals, indicating to keep track of originals
  • your client-side database writes happen inside your client-side method, which saves the old state to an originals tree prior to writing, which give you the optimistic ui
  • your client-side method is finished and the server-side method is called
  • livedata calls fetchOriginals, which prepares the originals tree data to be diffed when the server-method comes back
  • livedata processes the updates and patches the state
39

@sashko, are you able to chime in on the issue. I’d like to see if I can leverage livedata's optimistic update mechanism to get optimistic updates with Redux only.

40

It’s just what i’ve used in my React Native app without DDP. That would be used for the crud operations and then you could use the DDP hooks (below) to take care of data that comes in over DDP. I was thinking you could just use the added, changed, etc events of the DDP and skip minimongo but I haven’t looked through all the projects source. I was just looking at DDP.on and thinking you could hook into that:

DDP.on('removed', function (message) {
  if (message.collection !== 'posts') return;
  dispatch(Actions.removePost(message.id));
});

and then for the clientside just use the callback

// createPost action

Posts.insert(data, function (err, res) {
  // use res here the same way you would with ajax
  if (err) {
    dispatch(Actions.addPostFailed(data.id));
  }
  
});
41

Ah okay, I also am working on a React Native app and use the ddp-client (not ddp.js) NPM package with a few modifications to sync to my Redux store and it works quite nicely. The implementation of their observers give you more details about what changed and so forth. I haven’t tried doing optimistic updates yet, mainly because the app I have is for controlling smart devices. It doesn’t make sense to show that the light has turned on… unless it really did haha.

1 Like