I have a concurrency issue in my test's arrange block, where I do create some files in a subfolder(IO exception on file access in some test case runs). I have a parameterized test and test cases seems to run in parallel despite [NonParallelizable] attribute.

The issue is observed on tests run in Test-Explorer of VS2019.

Is there a possibility to prevent a parallel execution of test cases for some tests, it would be nice, if another tests(not test cases) in the fixture could still run in parallel.

[TestFixture]
public class ClassToTest_Fixture
{
    [TestCase("SubFolder", new[] { "a", "b", "c", "d", "e", "f", "g", "h" })]
    [TestCase("SubFolder", new[] { "a", "b", "c", "d", "e", "f", "g", "h" })]
    [TestCase("SubFolder", new[] { "a", "b", "c", "d", "e", "f", "g", "h" })]
    [TestCase("SubFolder", new[] { "a", "b", "c", "d", "e", "f", "g", "h" })]
    [TestCase("SubFolder", new[] { "a", "b", "c", "d", "e", "f", "g", "h" })]
    [NonParallelizable] //Doesn't help
    public void TestMethod(string folder, string[] files)
    {
        #region Arrange
        var fldr = Path.Combine(TestContext.CurrentContext.TestDirectory, folder);
        if(!Directory.Exists(fldr))
        {
            Directory.CreateDirectory(fldr);
        }
        foreach(var fn in files)
        {
            File.Create(Path.Combine(fldr, fn));
        } 
        #endregion Arrange

        //Act
        //Assert
    }

    [Test]
    public void TestMethodCanBeRunInParallel( )
    {
    }
}

CodePudding user response：

Short answer: This doesn't work currently, see https://github.com/nunit/nunit/issues/3371

More...

Since the operation of ParallelizableAttribute is always in relation to the parallel status of higher level tests (fixtures, namespaces, assembly), you should specify in the question whether the attribute is used on the fixture, the assembly or a setupfixture in the same namespace. I'm assuming that there are must be some higher level setting, otherwise you would not need to use [NonParallelizable] here, but I don't know what that setting is. :-) I'll edit this if you clarify it.

A similar problem arises when you want the individual cases to run in parallel, when the higher-level setting is non-parallel. That's easily resolved using [Parallelizable(ParallelScope.Children)]``. m Unfortunately, the NonParallelizableAttributedoesn't take a scope argument and is simply a synonym for[Parallelizable(ParallelScope.Nonein)]`. I think that was a mistake in my original design and maybe it will be corrected by the current team in a future release.

Workarounds...

1. The simplest thing would be to make the fixture containing these methods non-parallelizable. This may or may not be possible, depending on the details of your application, whether you have parameterized fixtures, etc. You could also try [Parallelizable(ParallelScope.Fixture)] on the fixture. That's supposed to allow the fixture, but not its children, to run in parallel.

2. More complicated and maybe confusing to your team... Add another fixture class and mark it [NonParallelizable]. You can nest it if you like, but do not derive it from the first class, since that will cause all the contained tests to run twice. Copy the method into the new class along with all the cases, which may not be run in parallel.

If it were my problem, I'd lean toward the first method, even though there's a bit of a performance hit. Developer time is generally what you want to optimize in test-writing, at least until you actually measure a significant problem.

Then, you can wait to see what the next NUnit release brings. :-)

CodePudding user response：

test cases seems to run in parallel was a wrong assumption, which I have made intuitively. After I have added an output with a time stamps and Thread.CurrentThread.ManagedThreadId I have found that testcases do run non parallel even without [NonParallelizable] attribute.

The issue was (my bad), that I haven't payed enough attention to the return value of File.Create(fileName), which is a FileStream.
Since I have not disposed a FileStream I had issues (IO exceptions) with next test case run.

Adding FileStream.Close() has solved the issue:

...
File.Create(Path.Combine(fldr, fn))?.Close();
...