Problems handling calls to mongo on Meteor 3.0-alpha.15

I’m converting some custom packages to Meteor 3.0-alpha.15. This simple mocha test works in alpha-15 server-side but not client-side. In Meteor 2.13.3 it works both client-side and server-side:

/* eslint-env mocha */

import assert from 'assert'
import { Mongo } from 'meteor/mongo'
import { Meteor } from 'meteor/meteor'
import { Random } from 'meteor/random'

const collection = new Mongo.Collection('test_mocha', {})

describe('testMocha', function () {
  it('Should be insert a document', function (done) {
    const allow = {
      insert: function () {
        return true
      },
      update: function () {
        return true
      },
      remove: function () {
        return true
      }
    }

    if (Meteor.isFibersDisabled) {
      Object.assign(allow,
        {
          insertAsync: function () {
            return true
          },
          updateAsync: function () {
            return true
          },
          removeAsync: function () {
            return true
          }
        })
    }

    // Full permissions on collection
    collection.allow(allow)

    const key = `test_${Random.id()}`
    collection.insertAsync({ _id: key, test: 1 })
      .then(() => collection.find({ _id: key, test: 1 }).countAsync())
      .then((docs) => {
        assert.equal(docs, 1)
      })
      .then(() => done())
      .catch(done)
  })
})

The error is AssertionError [ERR_ASSERTION]: 0 == 1. The reason seems to be that the processing of the ‘replace’ message, due to the invocation of the insertAsync method, is launched after the document has been inserted into the local collection, effectively deleting the previously inserted record.

mongo/collections.js

...
// place the whole doc" message coming from the quiescence
// of method writes to an object? (Note that 'undefined' is a valid
// value meaning "remove it".)
if (msg.msg === 'replace') {
  var replace = msg.replace;
  if (!replace) {
     if (doc) await self._collection.removeAsync(mongoId);
   } else if (!doc) {
      await self._collection.insertAsync(replace);
   } else {
....

At the end of the insert operation, the collection is empty. If I replace the insertAsync method with insert, it works.

@grubba, Do you know if this is a problem already solved in the next version? Thanks.

This test also does not work, client-side it returns a promise but should return information on inserted or updated documents.

  if (Meteor.isClient) {
    it('Test update', function () {
      const collection2 = new Mongo.Collection('test_mocha', { connection: null })

      const ret = collection2.upsert({ _id: 'test' }, { $set: { test: 1 } })
      assert.equal(!!ret.then, false)
    })
  }

Is there a reason for distinguishing handles between synchronous and asynchronous in the management of permissions on remote collections (allow-deny)? Now one is forced to write:

collection.allow({
insert: ...
insertAsync: ...
})

Wouldn’t it be a good idea to handle this with just one handle?

It is sad not to have feedback or a discussion on concrete issues concerning the new version. I am working on commonly used packages such as redis-oplog, collection2, collection-hooks. I have already asked this question in other threads: do we use the forum for these things or do we open issues in the project? I am struggling to work out whether they are known problems, misinterpretations or unfinished modules. We are all working for the cause!

@grubba @denyhs

I could be wrong but these kind of test always run on server, not on client. You need something like cypress to run code on real browser environment.

I think you are either mistaken or not familiar with remote collections. These have been present since the birth of Meteor, they allow calls from the client to persistent mongo collections to be handled transparently. No need for Cypress, just use mocha. These are tests that work in all versions on established packages.

We have been working with Meteor since its first release and have applications with thousands of users in SaaS deployment.

I know that feature, that’s why I asked you for insecure package. But in other thread, you said you don’t use that package. It’s interesting, I wonder is there another package which does the same?
I personally don’t use that insecure package.

I confirm that I never use the insecure package, which is not intended for the packages in question. These are the packages used in running the tests:

meteor@2.0.0-alpha300.15                  # Shared foundation for all Meteor packages
static-html@1.3.3-alpha300.15             # Define static page content in .html files
standard-minifier-css@1.9.3-alpha300.15   # CSS minifier run for production mode
standard-minifier-js@3.0.0-alpha300.15    # JS minifier run for production mode
es5-shim@4.8.1-alpha300.15                # ECMAScript 5 compatibility for older browsers
ecmascript@0.16.8-alpha300.15              # Enable ECMAScript2015+ syntax in app code
typescript@4.9.5-alpha300.15              # Enable TypeScript syntax in .ts and .tsx modules
shell-server@0.6.0-alpha300.15            # Server-side component of the `meteor shell` command
webapp@2.0.0-alpha300.15                  # Serves a Meteor app over HTTP
mongo@2.0.0-alpha300.15

As you can see, this is a clean environment.

If you don’t use insecure package, then Meteor enabled it by default? That’s interesting and dangerous.

No, the insecure package has nothing to do with it, it is not a subscription problem. The problem lies with the management of remote collections and the interpretation/timing of the return of methods dedicated to this.

I think it’s another package named autopublish.

I repeat that it is not a publishing problem, but in the management of remote collections and in the retrieval of the server response by the client.

Hopefully someone from Meteor can give their feedback here. Currently the work on actual code probably takes up the majority of their time, but as we get closer to a beta release the number of issues cropping up is sure to jump significantly. Probably the best places to discuss such issues is the forum itself and github.

Once a beta is released it would be great if some of the resources would be assigned to documentation as well. The lack of a migration guide will then start to hurt much more.

1 Like

Hi @pbeato, thanks for reporting this, and sorry for the delay in answering you.

We were not aware of this, and it seems serious. We have lots of tests for this type of scenario, so I’m not sure why we didn’t catch this one yet, but I’ll add this to our sprint so I can take a look at this.

I’ll keep you informed through this thread.

1 Like

Thank you @denyhs, could you also clarify this scenario for me? In previous versions of the insert, update, … mongo methods, you could pass a callback, which was useful on the client side to get feedback about the outcome of the server-side call. Now, in the asynchronous version, this is no longer possible. How is it possible to intercept any errors in the call to the method published in allow-deny?

As of today, the solution we have is to provide some flags when calling these functions from the client. When you call some Mongo method from the client these flags are passed down and used here. This means, right now, in version 3.0 when you wait for the result of a updateAsync, for example, you’re actually waiting for the server finish its processing.

Will this always be like this? Maybe no. We already have discussions around this that we plan to address when we start working on the Beta version.

About the test that is failing, I tried to reproduce it, and for me, it is passing… Could you provide a repository with these tests and the command you’re using to run them?

here the repository to reproduce the error. The error only occurs on the client side, the server side works. The command is meteor npm run test:browser.

Hi @denyhs, I checked the problem with Meteor 3.0 alpha-16 but I have the same problem. I added an observe to the collection to monitor the operations, and actually received an add and a remove. If it helps, if I add another collection.insertAsync({ _id: key, test: 2}) before the statement everything seems to work.

Thanks for the info! This is on my backlog, and I’ll get to it still this week.

Hi @pbeato, to give you an update on this, I know what’s happening.

As I explained here:

You can provide some flags depending on the scenario you want. In this case, you want the stub result and don’t want to wait for the server to finish. So you can use the flag returnServerResultPromise as false when calling a Meteor.callAsync, Meteor.applyAsync, or in this case, Collection.[method]Async.

So, if you change:

collection.insertAsync({ _id: key, test: 1 })

to:

collection.insertAsync({ _id: key, test: 1 }, { returnServerResultPromise: false })

The client-side test should pass now.

We’re definitely going to improve this. And it should happen soon, as the task to fix this is already in progress.

2 Likes

Hi @denyhs, I come back to this thread. I tried with the 3.0-beta.0 and the problem seems to persist, do you have any idea how to solve it? I’m trying to migrate the meteor-collection-hooks package but this issue is stopping me. I know you are changing the semantics of the call, should I take into consideation other parameters in the call?