EFCore throws NullReferenceException on complex splitQuery, but only sometimes-CodePudding

I have an EFCore query that, for the same parameter values and same database state, sometimes throws a NullReferenceException. Any ideas how I can identify where the problem lies?

The query looks like this (yes it's a somewhat complicated db structure):

var lines = _context.Calcs.Where(x => x.PlanId == planId && x.IsDeleted == false)
    .Include(x => x.Taggings)
    .ThenInclude(x => x.CalcTag)
    .Include(x => x.Destination)
    .ThenInclude(x => x.FormatCode)
    .Include(x => x.Destination)
    .ThenInclude(x => x.Dimensions.Where(vtd => vtd.PlanId == planId))
    .Include(x => x.CalcDimensions)
    .ThenInclude(x => x.Dimension)
    .Include(x => x.CLVT)
    .ThenInclude(x => x.ValueType)
    .Include(x => x.CLVT)
    .ThenInclude(x => x.CLVTJoinFields)
    .ThenInclude(x => x.Dimension)
    .Include(x => x.CLVT)
    .ThenInclude(x => x.CLVTTagFields)
    .Include(x => x.CLVT)
    .ThenInclude(clvt => clvt.PreCalc)
    .Include(x => x.CLVTPreCalcs)
    .ThenInclude(x => x.CLVTJoinFields)
    .Include(x => x.CLVTPreCalcs)
    .ThenInclude(x => x.CLVTTagFields)
    .AsSplitQuery()
    .ToList();

For exactly the same planId and database state, sometimes when this executes it results in a NullReferenceException and sometimes it doesn't. It looks like the call stack is always the same when the exception is thrown, which is helpful.

Exception is:

System.NullReferenceException
  HResult=0x80004003
  Message=Object reference not set to an instance of an object.
  Source=Microsoft.EntityFrameworkCore.Relational
  StackTrace:
   at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.Enumerator.Dispose()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at MyClass.MyMethod(Int32 planId) in C:\code\server\MyProduct\MyClass.cs:line 94

Searching for other NullReferenceException cases from EFCore it seems like it can happen when there's a null value in the database when the EF model would expect it to be non-nullable. But in this case the same data is being pulled back and often there's no exception, so I'm guessing that's not the case here. What else could cause this?

My next guess is that in the code path that causes the problem something else has happened with the DbContext, e.g. it's cached some objects, causing a different behavior for this same query. But my _context.ChangeTracker.DebugView.ShortView is empty when this exception is thrown, suggesting it's not that. My context usually has QueryTrackingBehavior==NoTracking and AutoDetectChangesEnabled==false unless it's in the middle of an update, but I tested without that (i.e. default tracking behavior) and got the same results.

I've used SQL Server extended events to trace the SQL that's executed. I see about 5 queries being started and I think the exception happens before rpc_completed fires for them all. I haven't yet analysed this thoroughly enough to know if it always throws the exception at the same point with receiving the SQL server results. But given that the results of the queries are the same each time I'm not sure that analysing this further is going to help much.

Any suggestions what could be causing the exception?

I'm using EFCore 5.0.12 (I was on 5.0.5 but upgraded and same problem happens)

<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" 
 Version="5.0.12" />

UPDATE:

Now I think that there's multiple threads using the same DbContext at once. They're all from the same request but there's some code a bit like this:

foreach (var x in y) 
{ 
    tasks.Add(MyAsyncMethodThatUsesDbContext(planId, otherParams)); 
} 
var z = await Task.WhenAll(tasks);

It's a fair bit more convoluted so not as obvious, but I'll see if it's that.

CodePudding user response：

The problem was caused by multiple threads using the same DbContext, which is a naughty thing to do. My DbContext wasn't shared by multiple requests but within a single background Hangfire job processing there was code that created a series of Tasks and then called WaitAll on them, something like this:

foreach (var x in y) 
{ 
    tasks.Add(MyAsyncMethodThatUsesDbContext(planId, otherParams)); 
} 
var z = await Task.WhenAll(tasks);

I thought there was something in EFCore that warned when you use the same DbContext from multiple threads, but perhaps it doesn't work either when running from a Hangfire job (i.e. no active asp request) or when the tasks are created all from the same thread?

One solution is to not parallelize the database access. Or alternatively ensure a separate DbContext is used for each task.