Pub-Sub data without mongoDB is not working

[ankibalyan]

I’m trying to implement a basic chat messaging server without storing the messages in MongoDB.
This is just to test the functionality, how it works, Here is my implementation.

if(Meteor.isServer){
    export const Messages = new Meteor.Collection("messages", {connection: null});
}

//Publish function

if(Meteor.isServer){
  Meteor.publish('messages', function(rid = 'ankit') {
    let self, finder, finderHandle;
    if (!this.userId) {
      return this.ready();
    }
    
    check(rid, String);
    finderHandle = Messages.find({});

    //
    finderHandle.observe({
      added: function (id, fields) {
        this.added( 'messages' , id, fields )
      }, // Use either added() OR(!)  ()
      changed: function (id, fields) {
        this.changed( 'messages' , id)
      },
      removed: function (id){
        this.removed( 'messages' , id, fields )
      }
    });

    this.ready();
  });
}

sendMessage Method

if (Meteor.isServer) {
	Meteor.methods({
		sendMessage: function (message){
			check(message, {
				msg: String,
				rid: String
			});

			if (! this.userId) {
				throw new Meteor.Error('not-authorized');
			}

			let save = Messages.insert({
				msg: message.msg,
				rid: message.rid
			});

			console.log("message saved")
			console.log(message);
			return {
				success: true
			};
		}
	});
}

and on client-side, I’m trying receiving the update but nothing seems to come.

if(Meteor.isClient){
  let Messages = new Mongo.Collection('messages')

  var user = {username: 'anil'};

  Meteor.loginWithKey(user, function (res) {
    var message = {
      rid: 'ankit',
      msg: 'Hi! This is for web client'
    };

    const handle = Meteor.subscribe('messages');

    Tracker.autorun(() => {
      const isReady = handle.ready();
      console.log(`Handle is ${isReady ? 'ready' : 'not ready'}`);
      console.log(handle);
    });
    Meteor.call('sendMessage', message);
  });
  console.log(Messages.findOne())
}

On the server-side, I’m getting this Error

Exception in queued task: RangeError: Maximum call stack size exceeded

Login / Sending message Methods are working fine. I’m not sure what I’m missing here, Though I’m able to do all this if I use MongoDB.

Thanks in advance, any comment will be appreciated.

[robfallows]

This: {connection: null} means don’t synchronise data between client and server.

What I think you want here is a “roll-your-own” publication. This may help (it discusses polling a REST endpoint, but the principle applies):

Best Way to handle a 3rd Party endpoint

This is relatively straightforward. You set up a client-only collection and a subscription in the usual way. RestData = new Mongo.Collection('rest-data'); // client only Meteor.subscribe('rest-docs'); On the server, you poll the REST endpoint at a suitable interval within a roll-your-own publication. Something along the lines of: Meteor.publish('rest-docs', function() { var init = true; var self = this; Meteor.setInterval(function() { var data = HTTP.get('some url').data; // for J…

[ankibalyan]

Thank you @robfallows I think there should be a way of sending the message from a method call to the other clients without the using Database.

This {connection: null} is for creating a minimonog instance i.e in memory data. So it should be able fetch the data from minimongo on the server and added it to current publication queue.

If I will be getting messages from a REST API. There are following concequences:

• Messages won’t get deliver in Real time there will be a time gap, It can’t be as Robust it should be.

• I’ve to depend on other server, there may be chances or getting failure if one server is down.

• I’ve to save the data in any of the DB, which I don’t want. If needed I will save in meteor’s mongo it self.

[robfallows]

You can send the results of a method call back to the calling client without any effort. If you want to “send the message from a method call to the other clients without the using Database.” then you will have to use pub/sub, and the technique outlined in that link is the way to do it.

[ankibalyan]

Hi, @robfallows I’ve tried altering that code, and now I’m able to get messages from one client to another client in real time. using the following update.

  Meteor.publish('messages', function(rid) {
    if (!this.userId) {
      return this.ready();
    }

    let finderHandle = Messages.find({
      $or: [
        {username: Meteor.users.findOne({_id: this.userId}).username},
        {rid: Meteor.users.findOne({_id: this.userId}).username}
      ]
    })

    finderHandle.observeChanges({
      added: (_id, record) => {
        this.added( 'messages',_id, record )
      },
      changed: (_id, record) => {
        this.changed( 'messages', _id, record)
      },
      removed: (_id, record) => {
        this.removed( 'messages',_id, record)
      }
    });

    this.onStop(() => {
      console.log("disconnected")
      // finderHandle.close(); // Giving an Error: finderHandle.close() function not found.
    })

    this.ready();
  });

Now the problem here is, when I disconnect and connect again, It is sending me all the previous messages as well. I want to remve that message as soon as it is delivered.
how can I handle this.removed event here?

also if I’m trying to use ‘this.onStop’ function finderHandle.close(); is giving me an error when a user disconnects saying this close function is undefined.

[robfallows]

Now I’m completely confused.

You said you wanted pub/sub without MongoDB and you have written something which seems to greatly complicate pub/sub with MongoDB.

[ankibalyan]

yes, but it I’ve given connection: null which isn’t storing it to DB.

[ankibalyan]

@robfallows Please correct me if I’m wrong, what exactly it is happening behind the scenes.

[karlitos]

Hey, I’m struggling with exact same problem. Btw. similar question was already asked here but there is no clear answer or complete example code to this. And the documentation is also probably little bit misleading.

What is the difference (in the impact on the app) between

const Messages = new Meteor.Collection("messages", {connection: null});

and

const Messages = new Meteor.Collection(null);

What is to way to achieve working pub/sub mechanism based on collections/MongoDb without the actual database write owerhead ?

[doctorpangloss]

There is no difference in performance. But you cannot return the cursor in a sub from the second example, because the magic that makes that work requires a named collection.

[karlitos]

Many thanks for your explanation. I tried to reproduce the example from@ ankibalyan

// imports/api/presentations/presentations.js
// Creates a new Mongo collections and exports it
export const Presentations = new Mongo.Collection('presentations', {connection: null});

if (Meteor.isServer) {
	Meteor.publish('presentationByMapId', function (mapId) {

		const subscriptionHandle = Presentations.find({ matchingMapId: mapId });

		subscriptionHandle.observeChanges({
			added: (_id, record) => {
				this.added( 'presentations', _id, record);
			},
			changed: (_id, record) => {
				this.changed( 'presentations', _id, record);
			},
			removed: (_id, record) => {
				this.removed( 'presentations',_id, record);
			}
		});

		this.onStop(function () {
			subscriptionHandle.stop();
		});

		return this.ready();
	});
}

But on the client in my react code, the collection is still empty, although the handle.ready() is true.

import { Presentations } from '../api/presentations/presentations';

...

const TrackerWrapper = withTracker((props) => {

    ...

	const presentationHandle = Meteor.subscribe(PRESENTATION_BY_MAP_ID, props.currentMapId);

	const handles = [
		 
       ...

		presentationHandle,
	];

	// determine if any subscription is loading
	const loading = handles.some(handle => !handle.ready());
	
    ...

	const ongoingPresentation = presentationHandle.ready() &&
								currentMap &&
								currentMap.presentationMode === true ?
								Presentations.findOne({ matchingMapId: props.currentMapId }) : null;


	return {
		loading,
		...
		currentMap,
		ongoingPresentation,
	};
})(MapContainer);

[karlitos]

I am now completely clueless. The line Presentations.findOne({ matchingMapId: props.currentMapId }) call stays always undefined.

But in the network-tab I see messages about changes in the presentations collection appearing when I perform changes in other client.

And upon closing a tab with client subscribing to this publication I get error:

eeException in onStop callback: TypeError: subscriptionHandle.stop is not a function

[j18ter]

Can’t comment on your attempts to get pub/sub working without MongoDB with the help of the low-level pub/sub API you proposed, but wanted to mention that we use the standard pub/sub approach without even running a MongoDB database, and it works fine. This did require a small number of adjustments in Meteor’s mongo package.
We changed 4 places in packages/mongo/collection.js. Our changes are enabled by setting the environment variable MONGO_URL to the string “none”.

1. Force options.connection to null in the Collection constructor, unless called on the client, i.e.,

  if (!Meteor.isClient && process.env.MONGO_URL === "none") {
    options.connection = null;
  }

2. Ignore attempts to create an index in function _ensureIndex, if MONGO_URL is “none”, unless running on the client.
3. Comment out the throw in function createIndex, if MONGO_URL is “none”.
4. Do nothing in _dopIndex, if MONGO_URL is none.

This way we can choose to use MongoDB by setting MONGO_URL as usual, or just set it to “none” and all collections are in-memory collections only, and both work with the same JavaScript source code. The main reason we needed to change the mongo package was to avoid trouble when built-in Meteor packages create their own collections and indexes, e.g., the accounts package. If one isn’t using such packages, modifying mongo isn’t required, one can simply pass the null connection when creating them, but our modified mongo package is also convenient because we do not need to pass {connection: null}, and the same source code works with or without running a database, controlled by MONGO_URL.