What's the point of running unit tests in random order?-CodePudding

I noticed the default behavior of Karma for my Angular builds is to run the Jasmine unit tests in random order. What is the benefit of running your tests in random order, vs running them in the same order every time?

CodePudding user response：

Sometimes tests modify state that doesn't get reset for every test run. Maybe a test modifies a global variable that is shared by all tests. Maybe a test writew something to the database and doesn't clean it up when the test is done. These things shouldn't happen, but sometimes they do happen.

For example:

// Stupid contrived example. Don't ever do this.

let num = 0

it('A', () => {
  expect(num   1).toEqual(1)
})

it('B', () => {
  num = 10
  expect(num   2).toEqual(12)
})

it('C', () => {
  expect(num   3).toEqual(13) // passes if B is run before C, otherwise fails
})

This may work fine if you always run test A, then B, then C. But then strange things my happen if you run only test C. Because of the changes that test A and B made, C passes when you run them all together. But then you run test C alone and it fails.

And now you are staring at you terminal output dumbfounded muttering "how on earth can test C fail?! It just passed!"

Randomizing the order helps with this a bit. Each test should be totally isolated and it shouldn't matter what order they run in. Randomizing them may help uncover this badness.

So, given that the order shouldn't matter, then why not randomize the order?