How to store an EJSON custom type in the database?


#1

I created a custom EJSON data-type. Transfer from client to the server works as expected, the server constructs the correct data-type. However, if I store an object into the database, it is always stored as a plain object. Mongo does not store any additional metadata about the type as I would have expected.

To make sure that my custom code is not interfering, I took the sample code from here:

and just tried to store an Address object to a Mongo collection, using:

var myAddress = new Address("San Francisco", "CA");
var addresses = new Mongo.Collection('addresses');
addresses.insert(myAddress);

But this resulted in: Error: Only plain objects may be inserted into MongoDB

I always thought that storing custom data types right inside MongoDB was one of the central use-cases of EJSON?


#2

That example on fact involves saving the custom type onto mongodb and retreiving it back to verify the type casting.

You probably have something missing in your code. Would you mind sharing it?


#3

The code is pretty simple, it just adds the above lines to the sample code of the linked page:

function Address(city, state) {
  this.city = city;
  this.state = state;
}

Address.prototype = {
  constructor: Address,

  toString: function() {
    return this.city + ', ' + this.state;
  },

  // Return a copy of this instance
  clone: function() {
    return new Address(this.city, this.state);
  },

  // Compare this instance to another instance
  equals: function(other) {
    if (!(other instanceof Address))
      return false;

    return this.city == other.city && this.state == other.state;
  },

  // Return the name of this type which should be the same as the one
  // padded to EJSON.addType
  typeName: function() {
    return "Address";
  },

  // Serialize the instance into a JSON-compatible value. It could
  // be an object, string, or whatever would naturally serialize
  // to JSON
  toJSONValue: function() {
    return {
      city: this.city,
      state: this.state
    };
  }
};

// Tell EJSON about our new custom type
EJSON.addType("Address", function fromJSONValue(value) {
  // the parameter - value - will look like whatever we
  // returned from toJSONValue from above.
  console.log(value);
  return new Address(value.city, value.state);
});

var myAddress = new Address("San Francisco", "CA");
var addresses = new Mongo.Collection('addresses');
addresses.insert(myAddress);

My own code is a bit more complex, as it also involves Collection2 schemas. And it doesn’t throw an error (as the above code does), yet the data in the database is still just a plain object.


#4

I can’t reproduce an error and the code seems to work.

Check this out: http://meteorpad.com/pad/caXGoJ39cMuWbZcs8/Custom%20EJSON%20Type%20in%20Mongo

Open up the app url in a new window an fire up the console. You’ll see that the type is of Address as expected.

If you look at your database with an external mongodb client, though, you will see no indication that it is an ejson enhanced object because to mongodb, it is just a regular object (in fact just city and state keys embedded in an address key) and the type casting is done within the meteor application environment.


#5

Thanks for providing this Meteorpad.

Maybe I’m dumb, but in the console [of the app] (http://app-95czarmf.meteorpad.com/) I see an object of type “Object”, not “Address”:

Object { _id: "RovYPbxGkoAxMofyM", name: "Chris Mather", createdAt: Date 2015-09-28T10:57:09.766Z, file: Uint8Array[5], address: Object }

I’ve tested this in both Firefox and Chrome, though I don’t think the browser makes any difference.

I could not even think of how this should work, i.e. how the Meteor layer should know what type of object Mongo returns, if there’s no additional metadata stored in the database. I would have expected that Meteor stores its $type information, but obviously it does not.

For a document-based database, it would not even be sufficient to store the type information with the Collection object, since a collection could contain data of any type. So it has to store the type information in the collection itself somehow.


#6

There IS metadata stored.
In the db, the data is like that :

{
      "EJSON$type" : "Address",
      "EJSON$value" : {
                         "EJSONcity" : "San Francisco",
                         "EJSONstate" : "CA"
                      }

}

I have used EJSON a little bit, but i was not convinced by it.
It makes your data tightly bind to meteor, i prefer to write my own serialization/deserialization methods.


#7

@waldgeist Here’s what I get:

@vjau what really??? I thought EJSON was smart enough with serialization and type conversion such that it did not require that kind of verbosity? Wow!

I had never explicitly used it, but was considering, and it is now dead to me exactly for this reason “data tightly bind to meteor

:frowning:


#8

Yikes!


#9

OK, I guess http://astronomy.meteor.com/#custom-types would now prove to be a very good choice in modelling within Meteor.

Astro.createType({
  name: 'type',
  constructor: function Type(fieldDefinition) {},
  getDefault: function(defaultValue) {},
  cast function(value) {},
  needsCast function(value) {},
  plain function(value) {}, // its job is to convert a value of your type to a plain JavaScript value. This plain value will be stored in the database.
  needsPlain function(value) {}
});

#10

@serkandurusoy: Strange that you get a completely different object than me. At the moment, Meteorpad is down for me, so I cannot cross-check. How did you get access to the underlying database?

Anyway: Thanks for pointing me to the Astronomy package. I was not aware of it. I only knew socialize:base-model, but this seemed to be in a rather early development stage.

To be honest, I would expect something like Astro from the base framework. Having a package like Astronomy is great, but what if the maintainer loses interest? Having a good ODM mapping framework is such a key functionality that I am wondering why MDG does not provide it.

Anyways. Still wondering how you got this Address sample working. :smile:


#11

How did you get this data into the database? When I tried to, I only got an error message, as described in my initial post.


#12

I downloaded the app from meteorpad and ran it on my computer. BTW, meteorpad is not down, you should use the meteorpad link not the app preview link since it changes across restarts.

ORM in core is an on the roadmap but it is a tricky subject. Not everyone favors ORM’S since they are an abstraction layer and abstractions carry their own set of (sometimes very frustrating) cons. And it definitely is not something you need/want in every app.


#13

Yes, you’re right, ORM shouldn’t be a must. But it should be available for those who need it.

BTW: After reading a bit throught the Astronomy docs, I am already loving this package. Having known this package last week would have saved me lots of frustration trying to accomplish the same thing on my own. I already implemented some kind of type management and serialization (and struggled with adding EJSON compatibility), but I will switch this to the Astronomy package for sure. So much easier if you can use an existing framework.

The only thing I am missing is an integration with aldeeds AutoForm package. But I read that Jagi is already working on a kind of Astronomy AutoForm.


#14

I remember now, i had same problem as you with EJSON.
EJSON doesn’t allow you to store a custom object “as-is” in the db.
It can only be a subtype of another object.
With your example, you must have let’s say a client plain object like this with an Address custom type

var address = new Address("San Francisco", "CA"):
var client = {
  name : "doe",
  address : address
};

It was one of the reasons that made it not so useful.


#15

Thanks for sharing this. This is indeed a huge limitation. Did not know this. Can’t remember the docs are stating this, either.