What approach would you recommend for creating a news feed from users' posts?

[bluray]

Suppose you have multiple users, who can create posts on their own profiles.
If a user goes to his news feed, he’ll see the latest posts from his contacts (yes, similar to fb/insta).

I can think of only three approaches:
A) To know what posts to display, the server can check all the recent posts from the user’s contacts, then sort them by date and return with a limit.
B) Every post from every user is indexed by date, then when a user opens his news feed, the server checks all those documents to see if the users are actually contacts and the posts should be considered or ignored.
C) Every time a user posts to his profile, an id of the post goes to the news feed of every single one of his contacts.

Option C is the one I like the most for now.
Every user would have a doubly-linked list of post blocks, where those post ids would be added.
Eg: find a contact, get his latest post block _id (a document that contains 20 post ids). If the block contains less than 20 posts, add it there, otherwise create a new block, add it there and set it as the head of the linked list.

My guess is that you’d recommend something more professional and complex, and I’d love to know what that would be. Thanks a lot!

[alawi]

Actually, custom newsfeed is not trivial to create due to scaling/performance challenges. But I think option C is on the right track if the feed to be shown can be prepared in advanced by adding those IDs, this will ensure quick retrieval of the feed data.

I’d recommend going the following slides to better understand the scaling challenges of custom newsfeed. The other option is to outsource the feed creation to a third party like https://getstream.io/.

[doctorpangloss]

The easiest way to render something is to have a single Mongo document correspond to a single logical page. So a single Mongo document should correspond to a single user’s feed.

This may look something like

{_id: userId + ':feed',
items: [{
  from: nameOfPostingUser,
  createdAt: ...
  title: postTitle,
  body: ...
}, ...]

Easy peasy. Keep your data model extremely literal, simple and 1:1. Don’t overthink the paging, paging is a separate problem that everyone keeps relitigating over and over again on this forum. Paging a large document is a general process. Wait until you need paging before implementing it.

It should be clear that whenever a user posts a piece of content, you must have a way of determining who’s interested. That’s up to you, there’s no answer. Don’t sweat the scaling.

[storyteller]

The problem I see with option C is that if someone new subscribes you need to a user you need to add the relevant posts to their feed as well or they will see only the new posts. Depends on what you want.

@copleykj made a package for this: https://atmospherejs.com/socialize/feed

[znewsham]

Whats wrong with a mix of A + B, e.g., you index posts by userId and date? Something like

Meteor.publish("news", function(contactIds) {
   return Posts.find({ userId: {$in: contactIds}, date: { $gt: someRecentDate } }, { sort: { date: -1 }, limit: 10 });
});

You could also get the contactIds in the publication itself of course.

[alawi]

Ah yeah but I think that depends on the requirements. Do we want to show previous posts? It might make sense in something like facebook group but for that, you don’t really need personalized newsfeed, just query all posts for that group. But in the case of a personalized newsfeed do we want to show previous posts? am I missing something?