Why are Mongo ObjectID's not unique?-CodePudding

According to the MongoDB docs ObjectID's are suppose to be created by

ObjectID is a 96-bit number which is composed as follows:

a 4-byte timestamp value representing the seconds since the Unix epoch (which will not run out of seconds until the year 2106)

a 5-byte random value, and

a 3-byte incrementing counter, starting with a random value.

So when I do

const mongoose = require('mongoose');
const carSchema = new mongoose.Schema({ driver: mongoose.ObjectId });

const Car = mongoose.model('Car', carSchema);

const car = new Car();

let i = 0;
while (i < 1000) {
  car.driver = new mongoose.Types.ObjectId();
  console.log(car.driver.toString());
  i  ;
}

I expect to see the 3 different parts change. However what I see is only one part increment.

~/tmp$ head t2 
638f7d3f37664dec556b0491
638f7d3f37664dec556b0492
638f7d3f37664dec556b0493
638f7d3f37664dec556b0494
638f7d3f37664dec556b0495
638f7d3f37664dec556b0496
638f7d3f37664dec556b0497
638f7d3f37664dec556b0498
638f7d3f37664dec556b0499
638f7d3f37664dec556b049a
~/tmp$ tail t2 
638f7d3f37664dec556b086f
638f7d3f37664dec556b0870
638f7d3f37664dec556b0871
638f7d3f37664dec556b0872
638f7d3f37664dec556b0873
638f7d3f37664dec556b0874
638f7d3f37664dec556b0875
638f7d3f37664dec556b0876
638f7d3f37664dec556b0877
638f7d3f37664dec556b0878

Question

Can someone figure out why I don't get the 3 different parts change?

CodePudding user response：

Updated as question was clarified.

Use the following node.js code to test:

const mongoose = require('mongoose')
const carSchema = new mongoose.Schema({ driver: mongoose.ObjectId })
const Car = mongoose.model('Car', carSchema)
const car = new Car()

const splitElements = (id) => {
  return {
    seconds: parseInt(id.slice(0, 8), 16),
    random: parseInt(id.slice(8, 18), 16),
    counter: parseInt(id.slice(18, 24), 16)
  }
}

const sleep = (ms) => {
  return new Promise(resolve => setTimeout(resolve, ms))
}

const init = async () => {
  let i = 0
  while (i < 20) {
    car.driver = new mongoose.Types.ObjectId()
    console.log(car.driver.toString(), JSON.stringify(splitElements(car.driver.toString())))
    await sleep(100)
    i  
  }
}
init()

You will notice that I added a small sleep in there as well as artificially parsing the id into it's constituent elements.

If you run this program 3 times you get something like this

       FIRST TIME
638f9befbe5587e19f76c51c {"seconds":1670355951,"random":817478754719,"counter":7783708}
638f9befbe5587e19f76c51d {"seconds":1670355951,"random":817478754719,"counter":7783709}
638f9befbe5587e19f76c51e {"seconds":1670355951,"random":817478754719,"counter":7783710}
638f9befbe5587e19f76c51f {"seconds":1670355951,"random":817478754719,"counter":7783711}
638f9befbe5587e19f76c520 {"seconds":1670355951,"random":817478754719,"counter":7783712}
638f9befbe5587e19f76c521 {"seconds":1670355951,"random":817478754719,"counter":7783713}
638f9befbe5587e19f76c522 {"seconds":1670355951,"random":817478754719,"counter":7783714}
638f9befbe5587e19f76c523 {"seconds":1670355951,"random":817478754719,"counter":7783715}
638f9bf0be5587e19f76c524 {"seconds":1670355952,"random":817478754719,"counter":7783716}
638f9bf0be5587e19f76c525 {"seconds":1670355952,"random":817478754719,"counter":7783717}
638f9bf0be5587e19f76c526 {"seconds":1670355952,"random":817478754719,"counter":7783718}
638f9bf0be5587e19f76c527 {"seconds":1670355952,"random":817478754719,"counter":7783719}
638f9bf0be5587e19f76c528 {"seconds":1670355952,"random":817478754719,"counter":7783720}
638f9bf0be5587e19f76c529 {"seconds":1670355952,"random":817478754719,"counter":7783721}
638f9bf0be5587e19f76c52a {"seconds":1670355952,"random":817478754719,"counter":7783722}
638f9bf0be5587e19f76c52b {"seconds":1670355952,"random":817478754719,"counter":7783723}
638f9bf0be5587e19f76c52c {"seconds":1670355952,"random":817478754719,"counter":7783724}
638f9bf0be5587e19f76c52d {"seconds":1670355952,"random":817478754719,"counter":7783725}
638f9bf1be5587e19f76c52e {"seconds":1670355953,"random":817478754719,"counter":7783726}
638f9bf1be5587e19f76c52f {"seconds":1670355953,"random":817478754719,"counter":7783727}

       SECOND TIME
638f9bf79c0221dcd6dd3363 {"seconds":1670355959,"random":670050671830,"counter":14496611}
638f9bf79c0221dcd6dd3364 {"seconds":1670355959,"random":670050671830,"counter":14496612}
638f9bf79c0221dcd6dd3365 {"seconds":1670355959,"random":670050671830,"counter":14496613}
638f9bf79c0221dcd6dd3366 {"seconds":1670355959,"random":670050671830,"counter":14496614}
638f9bf79c0221dcd6dd3367 {"seconds":1670355959,"random":670050671830,"counter":14496615}
638f9bf79c0221dcd6dd3368 {"seconds":1670355959,"random":670050671830,"counter":14496616}
638f9bf79c0221dcd6dd3369 {"seconds":1670355959,"random":670050671830,"counter":14496617}
638f9bf79c0221dcd6dd336a {"seconds":1670355959,"random":670050671830,"counter":14496618}
638f9bf79c0221dcd6dd336b {"seconds":1670355959,"random":670050671830,"counter":14496619}
638f9bf89c0221dcd6dd336c {"seconds":1670355960,"random":670050671830,"counter":14496620}
638f9bf89c0221dcd6dd336d {"seconds":1670355960,"random":670050671830,"counter":14496621}
638f9bf89c0221dcd6dd336e {"seconds":1670355960,"random":670050671830,"counter":14496622}
638f9bf89c0221dcd6dd336f {"seconds":1670355960,"random":670050671830,"counter":14496623}
638f9bf89c0221dcd6dd3370 {"seconds":1670355960,"random":670050671830,"counter":14496624}
638f9bf89c0221dcd6dd3371 {"seconds":1670355960,"random":670050671830,"counter":14496625}
638f9bf89c0221dcd6dd3372 {"seconds":1670355960,"random":670050671830,"counter":14496626}
638f9bf89c0221dcd6dd3373 {"seconds":1670355960,"random":670050671830,"counter":14496627}
638f9bf89c0221dcd6dd3374 {"seconds":1670355960,"random":670050671830,"counter":14496628}
638f9bf89c0221dcd6dd3375 {"seconds":1670355960,"random":670050671830,"counter":14496629}
638f9bf99c0221dcd6dd3376 {"seconds":1670355961,"random":670050671830,"counter":14496630

       THIRD TIME
638f9bfab8f928518038320f {"seconds":1670355962,"random":794454151552,"counter":3682831}
638f9bfbb8f9285180383210 {"seconds":1670355963,"random":794454151552,"counter":3682832}
638f9bfbb8f9285180383211 {"seconds":1670355963,"random":794454151552,"counter":3682833}
638f9bfbb8f9285180383212 {"seconds":1670355963,"random":794454151552,"counter":3682834}
638f9bfbb8f9285180383213 {"seconds":1670355963,"random":794454151552,"counter":3682835}
638f9bfbb8f9285180383214 {"seconds":1670355963,"random":794454151552,"counter":3682836}
638f9bfbb8f9285180383215 {"seconds":1670355963,"random":794454151552,"counter":3682837}
638f9bfbb8f9285180383216 {"seconds":1670355963,"random":794454151552,"counter":3682838}
638f9bfbb8f9285180383217 {"seconds":1670355963,"random":794454151552,"counter":3682839}
638f9bfbb8f9285180383218 {"seconds":1670355963,"random":794454151552,"counter":3682840}
638f9bfbb8f9285180383219 {"seconds":1670355963,"random":794454151552,"counter":3682841}
638f9bfcb8f928518038321a {"seconds":1670355964,"random":794454151552,"counter":3682842}
638f9bfcb8f928518038321b {"seconds":1670355964,"random":794454151552,"counter":3682843}
638f9bfcb8f928518038321c {"seconds":1670355964,"random":794454151552,"counter":3682844}
638f9bfcb8f928518038321d {"seconds":1670355964,"random":794454151552,"counter":3682845}
638f9bfcb8f928518038321e {"seconds":1670355964,"random":794454151552,"counter":3682846}
638f9bfcb8f928518038321f {"seconds":1670355964,"random":794454151552,"counter":3682847}
638f9bfcb8f9285180383220 {"seconds":1670355964,"random":794454151552,"counter":3682848}
638f9bfcb8f9285180383221 {"seconds":1670355964,"random":794454151552,"counter":3682849}
638f9bfcb8f9285180383222 {"seconds":1670355964,"random":794454151552,"counter":3682850}

As the pause is introduced, you can see the seconds portion of the ObjectID does indeed increment, it is even consistently incrementing between runs.

The random 5 bytes are also changing between runs. Not on each invocation of new ObjectID(), but between application starts. See code bson, specifically looking where it sets the PROCESS_UNIQUE IF NOT NULL! Eg, this happens once and once only, so we don't expect it to change if our application is still running.

if (PROCESS_UNIQUE === null) {
  PROCESS_UNIQUE = randomBytes(5);
}

Finally, the counter increments as expected. Starting on a random value.

So, in summary, they do change, but you have to go slowly (1-2 seconds), to notice a change in the time based elements, and application restarts are the only trigger for changing the random part.

Side note from comments: Keeping Track of the counter:

If you use new mongoose.Types.ObjectId(), that is just placeholder for the mongodb driver const {ObjectID} = require('mongodb'), which also points to the bson library, so they all use the same object.ts class to generate the ObjectID. BUT, this is all subject to change, as MongoDB ObjectID generation used to contain 4 elements, so you shouldn't really try to second guess the internal workings as they are internal. That being said the ObjectID class has a private static field for index, which is set to random on initialisation and when you create an ObjectID through the generate() method, it increments the index each time.

However, why you want to keep track of this, I don't know. If you wanted to keep track of it, I'd just use the splitElements method and parse it as binary unless you really want to overwrite the objectid.ts class in your node_modules to gain access to it.