I have a requirement to lockout a user if three failed attempts are made within 15 minutes. The account will be automatically unlocked after a period. Now I am passing the parameters - maximum attempt count, the lockout window duration and lockout period as parameters to the class that implements the functionality. Even with values like 2s or 3s for the parameters will result in the unit test suite execution to complete about 30 seconds.

Is there any specific method or strategies used in these scenarios to reduce the test execution time?

CodePudding user response：

Time is an input

If you don't consider time an input value, think about it until you do -- it is an important concept -- John Carmack

Arrange your design so that you can check the complicated logic of deciding what to do independently from actually doing it.

Design the actually doing it parts such that they are "so simple there are obviously no deficiencies". Check that code occasionally, or as part of your system testing - those tests are still potentially going to be "slow", but they aren't going to be in the way (because you've found a more effective way to mitigate "the gap between decision and feedback").

CodePudding user response：

There are a couple of options:

Use a Test Double and inject an IClock that a test can control.
Use a smaller time resolution than seconds. Perhaps define the window and the quarantine period in milliseconds instead of seconds.
Write the logic as one or more pure functions.

Pure functions are intrinsically testable, so let me expand on that.

Pure functions

In order to ensure that we're working with pure functions, I'll write the tests and the System Under Test in Haskell.

I'm going to assume that some function exists that checks whether a single login attempt succeeds. How this works is a separate concern. I'm going to model the output of such a function like this:

data LoginResult = Success | Failure deriving (Eq, Show)

In other words, a login attempt either succeeds or fails.

You now need a function to determine the new state given a login attempt and a previous state.

Evaluate login

At first I thought this was going to be more complicated, but the nice thing about TDD is that once you write the first tests, you realise the potential for simplification. Fairly quickly, I realised that all that was required was a function like this:

evaluateLogin :: (UTCTime, LoginResult) -> [UTCTime] -> [UTCTime]

This function takes a current LoginResult and the time it was made (as a tuple: (UTCTime, LoginResult)), as well as a log of previous failures, and returns a new failure log.

After a few iterations, I'd written this inlined HUnit parametrised test:

"evaluate login" ~: do
  (res, state, expected) <-
    [
      ((at 2022 5 16 17 29, Success), [],
        [])
      ,
      ((at 2022 5 16 17 29, Failure), [],
        [at 2022 5 16 17 29])
      ,
      ((at 2022 5 16 18 6, Failure), [at 2022 5 16 17 29],
        [at 2022 5 16 18 6, at 2022 5 16 17 29])
      ,
      ((at 2022 5 16 18 10, Success), [at 2022 5 16 17 29],
        [])
    ]
  let actual = evaluateLogin res state
  return $ expected ~=? actual

The logic I found useful to tease out is that whenever there's a test failure, the evaluateLogin function adds the failure time to the failure log. If, on the other hand, there's a successful login, it clears the failure log:

evaluateLogin :: (UTCTime, LoginResult) -> [UTCTime] -> [UTCTime]
evaluateLogin (   _, Success)          _ = []
evaluateLogin (when, Failure) failureLog = when : failureLog

This, however, tells you nothing about the quarantine status of the user. Another function can take care of that.

Quarantine status

The following parametrised tests is the result of a few more iterations:

"is locked out" ~: do
  (wndw, p, whn, l, expected) <-
    [
      (ndt 0 15, ndt 1 0, at 2022 5 16 19 59, [], False)
      ,
      (ndt 0 15, ndt 1 0, at 2022 5 16 19 59, [
        at 2022 5 16 19 54,
        at 2022 5 16 19 49,
        at 2022 5 16 19 45
      ],
        True)
      ,
      (ndt 0 15, ndt 1 0, at 2022 5 16 19 59, [
        at 2022 5 16 19 54,
        at 2022 5 16 19 49,
        at 2022 5 16 18 59
      ],
        False)
      ,
      (ndt 0 15, ndt 1 0, at 2022 5 16 19 59, [
        at 2022 5 16 19 54,
        at 2022 5 16 19 52,
        at 2022 5 16 19 49,
        at 2022 5 16 19 45
      ],
        True)
      ,
      (ndt 0 15, ndt 1 0, at 2022 5 16 20 58, [
        at 2022 5 16 19 54,
        at 2022 5 16 19 49,
        at 2022 5 16 19 45
      ],
        False)
    ]
  let actual = isLockedOut wndw p whn l
  return $ expected ~=? actual

These tests drive the following implementation:

isLockedOut :: NominalDiffTime -> NominalDiffTime -> UTCTime -> [UTCTime] -> Bool
isLockedOut window quarantine when failureLog =
  case failureLog of
    [] -> False
    xs ->
      let latestFailure = maximum xs
          windowMinimum = addUTCTime (-window) latestFailure
          lockOut = 3 <= length (filter (windowMinimum <=) xs)
          quarantineEndsAt = addUTCTime quarantine latestFailure
          isStillQuarantined = when < quarantineEndsAt
      in
        lockOut && isStillQuarantined

Since it's a pure function, it can calculate quarantine status deterministically based exclusively on input.

Determinism

None of the above functions depend on the system clock. Instead, you pass the current time (when) as an input value, and the functions calculate the result based on the input.

Not only is this easy to unit test, it also enables you to perform simulations (a test is essentially a simulation) and calculate past results.