Migrate to new document structure in mongo 3.6-CodePudding

I have to migrate data from a structure from

  {
    "_id": "some-id",
    "_class": "org.some.class",
    "number": 1015,
    "timestamp": {"$date": "2020-09-05T12:08:02.809Z"},
    "cost": 0.9200000166893005
  }

{"_id": {
  "productId": "some-id",
  "countryCode": "DE"
},
"_class": "org.some.class",
"number": 1015,
"timestamp": {"$date": "2020-09-05T12:08:02.809Z"},
"cost": 0.9200000166893005
}

The change that is in the new document is the _id field is replaced by a complex _id object (productId : String, country : String).
The country field is to be completed for the entire collection with a specific value - DE.
The collection has about 40 million records in the old format and 700k in the new format. I would like to bring these 40 million to this new form. I’m using mongo 3.6, so I’m a bit limited and I’ll probably have to use the aggregate functions to create a completely new collection, and then remove the old one.
I will be grateful for help on how to do it - how the query that will do it should look like and how to keep these migrated 700k documents.
What I have got so far:

db.productDetails.aggregate(
{$match: {_id: {$exists: true}}},
{$addFields: {"_id": {"productId": "$_id", "country": "DE"}},
{$project: {_id: 1, _class: 1, number: 1, timestamp: 1, cost: 1}},
{$out: "productDetailsV2"}
)

but this solution would only work if I didn't have 700k documents in the new form.

CodePudding user response：

Your query is in the right direction. You may want to modify the $match filter a bit to better catch the old type documents.

db.collection.aggregate([
  {
    $match: {
      "_id.country": {
        $exists: false
      }
    }
  },
  {
    $addFields: {
      "_id": {
        "productId": "$_id",
        "country": "DE"
      }
    }
  },
  {
    $project: {
      "_id": 1,
      "_class": 1,
      "number": 1,
      "timestamp": 1,
      "cost": 1
    }
  },
  {
    $out: "productDetailsV2"
  }
])

Mongo Playground