AggregateOption MongoDB how to iterate C# Mongodb.Driver-CodePudding

I am turning my List of an MongoDb aggregation pipeline to a BatchSize operation but I am unsure how to loop through the batches it after creating an option of BatchSize of 100000. I could have 800K in the list or I could have 64M.

I have the options as:

//Batch size
AggregateOptions aggOptions = new AggregateOptions();
aggOptions.BatchSize = 100000;

My pipeline is such as:

 var sample = (entityCollection.Aggregate(aggOptions)
                       .Match(dateFilterString)
                       .Project(projectString));

And what I have is:

await sample.ForEachAsync(doc =>
 {
    var test = doc.ToList();

    Console.WriteLine("### Count ###: "   test.Count());

  }
);

I do not want to iterate one by one in each batch but want to apply a if batch exists MoveNextAsync or hasNext() and apply something like this 8 times, is it possible? Or is the Select statement doing the same thing and doing a loop? What is most efficient?

var keys = (entityCollection.Aggregate()
    .Match(dateFilterString)
    .Project(projectString)
    .Limit(100000)
    .ToList())
 .Select(x => (x[2][0].ToString().Substring(15, 32))).ToArray();

CodePudding user response：

Internally ForEachAsync uses an IAsyncCursor<MyEntityType> to iterate through the batches, which can be obtained and iterated like this:

var sample = (entityCollection.Aggregate(aggOptions)
                       .Match(dateFilterString)
                       .Project(projectString));

var cursor = sample.ToCursorAsync();

while (await cursor.MoveNextAsync())
{
    IEnumerable<MyEntityType> batch = cursor.Current;
    // do whatever you need to do with the batch here
}