I am updating a list of transactions by saving the transaction into the database list, I do not want to have duplicate entries in the list so I use $addtoset

this is because the request can be fired multiple times and we want to make sure that any changes are idempotent to the database. the only catch now is that we want to only store the latest 20 transactions

this could be done with a $push $sort $slice but I need to make sure duplicate entries are not available. there was a feature request to mongo back in 2015 for this to be added to the $addtoset feature, but they declined this due to 'sets' not being in an order... which is what the $sort function would have been

I thought I could simply append an empty push update to the update object, but from what I understand, each update is potentially threaded and can lead to undesirable edits if the push/slice fires before the $addtoset

right now, the values are an aggregated string with the following formula timestamp:value but I can easily change the structure to an object {ts:timestamp, value:value}

Update: current code, not sure if it will work as intended as each operation maybe independent

          await historyDB
            .updateOne(
              { trxnId: txid },
              {
                $addToSet: {
                  history: {
                    ts: time,
                    bid: bid.value,
                    txid: trxn.txid,
                  }
                },
                $push: {
                  history: {
                    $each: [{ts:-1}],
                    $sort: { ts: 1 },
                    $slice: -10,
                  },
                },
              },
              { upsert: true },
            ).exec();

CodePudding user response：

Your query doesn't work, as you are trying to update history multiple times, which is not allowed in simple update document and raises error Updating the path 'history' would create a conflict at 'history'.

You can however subsequently update history field multiple times with aggregation pipeline.

await historyDB.updateOne(
  { trxnId: txid},
  [{
    $set: {
      history: {
        $let: {
          vars: {
            historyObj: {
              ts: time,
              bid: bid.value,
              txid: trxn.txid,
            },
            historySafe: { $ifNull: ["$history", []] }
          },
          in: {
            $cond: {
              if: { $in: ["$$historyObj", "$$historySafe"] },
              then: "$history",
              else: { $concatArrays: [ "$$historySafe", ["$$historyObj"] ] }
            }
          }
        }
      }
    },
  },
  {
    $set: {
      history: {
        $function: {
          body: function(entries) {
            entries.sort((a, b) => a.ts - b.ts);
            return entries;
          },
          args: [{ $ifNull: ["$history", []] }],
          lang: "js"
        }
      }
    },
  },
  {
    $set: {
      history: {
        $slice: [ "$history", -10 ]
      }
    }
  }],
  { upsert: true },
).exec()
        

As of MongoDB 6.0, the second $set stage, which provides sorting, can be replaced with $sortArray operator (see here).