I am trying something simple using the mongo-go-driver. I insert some datas in a collection, and I want them to be automaticaly deleted after a number of seconds.

I have read the following documentation : https://docs.mongodb.com/manual/tutorial/expire-data/#expire-documents-after-a-specified-number-of-seconds

Then I have wrote something in GO, but it does not seems to work as I expected. Maybe there is something I did not get, or I am doing the wrong way.

package main

import (
    "bytes"
    "context"
    "fmt"
    "log"
    "text/tabwriter"
    "time"

    "github.com/Pallinder/go-randomdata"
    "go.mongodb.org/mongo-driver/bson"
    "go.mongodb.org/mongo-driver/bson/primitive"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

func main() {
    ctx := context.TODO()

    client, err := mongo.NewClient(options.Client().ApplyURI("mongodb://localhost:27017"))
    if err != nil {
        log.Fatal(err)
    }
    err = client.Connect(ctx)
    if err != nil {
        log.Fatal(err)
    }

    db := client.Database("LADB")
    col := db.Collection("LACOLL")

    // add index to col
    // the goal is to set a TTL for datas to only 1 secondes (test purpose)
    model := mongo.IndexModel{
        Keys:    bson.M{"createdAt": 1},
        Options: options.Index().SetExpireAfterSeconds(1),
    }
    ind, err := col.Indexes().CreateOne(ctx, model)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(ind)

    // insert some datas each seconds
    for i := 0; i < 5; i   {
        name := randomdata.SillyName()
        res, err := col.InsertOne(ctx, NFT{Timestamp: time.Now(), CreatedAt: time.Now(), Name: name})
        if err != nil {
            log.Fatal(err)
        }
        fmt.Println("Inserted", name, "with id", res.InsertedID)
        time.Sleep(1 * time.Second)
    }

    // display all
    cursor, err := col.Find(ctx, bson.M{}, nil)
    if err != nil {
        log.Fatal(err)
    }
    var datas []NFT
    if err = cursor.All(ctx, &datas); err != nil {
        log.Fatal(err)
    }

    // I expect some datas not to be there (less than 5)
    fmt.Println(datas)
}

type NFT struct {
    ID        primitive.ObjectID `bson:"_id,omitempty"`
    CreatedAt time.Time          `bson:"createdAt,omitempty"`
    Timestamp time.Time          `bson:"timestamp,omitempty"`
    Name      string             `bson:"name,omitempty"`
}

CodePudding user response：

There's nothing wrong with your example, it works.

Please note that the expireAfterSeconds you specify is the duration after createdAt when the document expires, and that instant is the earliest time at which the document may be deleted, but there is no guarantee that the deletion will happen "immediately", exactly at that time.

Quoting from MongoDB docs: TTL indexes: Timing of the Delete Operation:

The TTL index does not guarantee that expired data will be deleted immediately upon expiration. There may be a delay between the time a document expires and the time that MongoDB removes the document from the database.

The background task that removes expired documents runs every 60 seconds. As a result, documents may remain in a collection during the period between the expiration of the document and the running of the background task.

Because the duration of the removal operation depends on the workload of your mongod instance, expired data may exist for some time beyond the 60 second period between runs of the background task.

As you can see, if a document expires, at worst case it may take 60 seconds for the background task to kick in and start removing expired documents, and if there are many (or the database is under heavy load), it may take some time to delete all expired documents.