Meteor with _id of type ObjectID generates wrong timestamps

help MongoDB
1

From Perplexity AI:

Q: Is there a known bug where the Meteor generated ObjectIDs do not have the correct date?

A: Yes, there is a known issue in Meteor where ObjectIDs generated by Meteor do not always have a meaningful or correct timestamp in their getTimestamp() value. This is because Meteor’s implementation of Mongo.ObjectID (when using { idGeneration: 'MONGO' }) historically generated ObjectIDs with random values, rather than encoding the current date and time as MongoDB’s native ObjectId does[5].

ObjectID values created by Meteor will not have meaningful answers to their getTimestamp method, since Meteor currently constructs them fully randomly.[5]

This means if you use Meteor’s Mongo.ObjectID to generate _id values, calling .getTimestamp() on those IDs may return a timestamp that does not correspond to the actual creation date of the document, and could even appear as a date far in the past or future.

Summary:

  • Meteor-generated ObjectIDs may not encode the correct creation timestamp.
  • This is a result of Meteor’s ObjectID implementation, not a bug in MongoDB itself.
  • If you need accurate timestamps, use a separate date field (e.g., createdAt) in your documents[5].

If you require ObjectIDs with correct timestamps, consider generating ObjectIds on the server using MongoDB’s native mechanism or always storing an explicit timestamp field.

Does this make sense??

I switched to ObjectIDs so I could drop the exact createdAt field and indexes I use in a lot of collections, and I can confirm the ObjectIDs do not encode a correct timestamp.

2

Yeah, that makes sense and lines up with what I’ve run into. Meteor’s Mongo.ObjectID doesn’t follow the standard ObjectID structure where the first 4 bytes represent a timestamp. If you’re relying on getTimestamp() for document creation times, it definitely won’t work as expected. Adding an explicit createdAt field still seems like the safest approach for now.

3

I have now mongo and mongo-id as local packages with the necessary updates, but I wonder why Meteor chose to generate those ids out of MongoDB standards.
@nachocodoner @storyteller any ideas on this?

A developer, when selecting { idGeneration: 'MONGO' } would expect these ids to be either generated by the Mongo driver or generated within the Mongo Standards. Otherwise it is a “looks like MONGO, feels like MONGO but doesn’t work like MONGO.”

The function I use now in mongo-id for id generation:

function generateObjectId() {
  const timestamp = Math.floor(Date.now() / 1000).toString(16).padStart(8, '0');
  const randomPart = Random.hexString(16);

  return timestamp + randomPart;
}
4

The only option to keep the timestamp true to your database is to generate it there. (Sure, server-database clock skew is going to be rather small, but it can vary with multiple containers.). You can do that by using rawCollection methods instead.

1 Like
5

Thanks a lot, Radosław.

This creates 2 perspectives:

  1. Use rawCollection and eventually create a new function (maybe 2, one for insertMany).
    In this case, all Meteor packages (such as Accounts) will be generating one type of Id since they are based on Collection.insertOneAsync while “code I write” will be generating a different kind of Id. This is not feasible.
if (!Mongo.Collection.prototype.insertNativeAsync) {
  Mongo.Collection.prototype.insertNativeAsync = function (doc) {
    const Collection = this.rawCollection()
    return Collection.insertOne(doc)
  }
}
  1. Overwrite the “original” insertAsync somewhere in the server side startup (maybe client side too) to use the rawCollection. This is basically what I would expect from { idGeneration: ‘MONGO’ } and now that you sent me this way, it seems to be a decent resolution for me.
Mongo.Collection.prototype.insertAsync = function (doc) {
    const Collection = this.rawCollection()
    return Collection.insertOne(doc)
  }

The original ObjectID generator in Meteor found in the mongo-id package contains a helper for reading time stamps:

getTimestamp() {
    return Number.parseInt(this._str.substr(0, 8), 16);
  }

This would only generate valid timestamps if the user controls the hex string used to generate these IDs. I see this is well-documented here Collections | Docs but I really don’t get the reasoning behind this apparently useless complexity and deviation from Mongo standard Ids.

If I came to Meteor from a Node + Mongoose or Node + MongoDB environment, this would be so confusing.

Perhaps an option such as { idGeneration: 'NATIVE_MONGO' } would be a good addition to the present options for those who want to adopt more compliancy with Mongo and improve performance.

Because I do not know/understand why Meteor doesn’t use Mongo ObjectIds for ObjectId creation, my confidence to go the rawCollection way is very low.

6

In its current form, Accounts won’t work with ObjectIds on the users collection – that’s because of the problems with their serialization, and because you can’t specify its ID generation method (or at least I don’t know if it’s possible). That can be fixed, though.

Sure, I get that. But it’s because Meteor does more than most (all?) backend-only frameworks – it can consistently generate IDs on both client and server. It’s required to make the client simulations and optimistic UI work.

I think it’d be possible only if no client-side simulations would need the ID (generating an ID could terminate the simulation) and if you’d be fine with a lack of latency compensation (currently Meteor server can generate the ID and send it to the client in parallel to the database operation). However, these two are exactly what the rawCollection().insertOne(...) is doing (as it doesn’t exist on the client and the server has to wait for the Oplog entry).

7

Thanks a lot. I feel a bit ashamed now discussing about these things with you :))) but maybe this will help others in understanding the _id within Meteor. Why I feel ashamed is because I used to know all these. When Meteor was taken over by Tiny, things felt a bit uncertain or unclear about the future of the stack, so I moved to doing more Node in Meteor and less Meteor.

With approval from the content owners, I re-published the original Meteor videos a couple of years ago, and all the answers are there: https://learn-meteor.netlify.app/

Thanks a lot for your time, now I feel confident, and I am almost 100% sure I am going to let the Accounts do their job with the String Ids, and I will be doing rawCollection inserts everywhere else.
In 1-to-1 relations such as User - Profile this is not that important but in 1 to many such as Discussion - Messages, ObjectIDs could provide that boost in sorting speed when you have millions if not billions of short messages in your DB.
Since I use no Optimistic UI or publications, I will try ObjectIDs with Users too, I will probably still generate them on the server but will use valid timestamps.

8

There’s no reason to ask and discuss! Such messages show that the documentation around it is not good enough, nothing more.