How to automatically delete every failing Cypress test?-CodePudding

I have inherited a large, out-of-date repo with a set of Cypress e2e tests that cumulatively take multiple days to run. I need to run the tests and then delete the ones that fail. Based on my experiences so far, around 80% of tests fail. So the scale of this task quickly becomes unwieldy manually.

Ideally the solution would be a single bash script. Another approach that comes to mind is to somehow export a list of failing tests to a CSV (which I haven't been able to figure out how to do either) and then delete every file in that list programmatically somehow. I'm using VSCode in case there are plugins that can help.

There is a secondary problem that simply running all tests quickly causes me to run out of memory. If there was some way to delete the tests as I go so that my entire task was accomplished by a single bash script, that would be amazing. But I can run the tests manually if this is not possible.

Right now I'm accessing the list of failed tests by just literally copying the terminal output to a text file. This would be easy enough to do programmatically, but the output doesn't even list the file names in an easy-to-extract manner. Example below (please overlook some weird formatting changes as I anonymized the file names for this post):

What is the best way to do this?

CodePudding user response：

export a list of failing tests to a CSV - When you run the tests via Cypress module API, you are running them in a node script with access to each test result and fs to write the results out.

Here's the basic concept

// e2e-run-tests.js
const cypress = require('cypress')
const fs = require('fs')

cypress
  .run({
    // the path is relative to the current working directory
    spec: './cypress/integration/**/*.spec.js',
  })
  .then((results) => {
    console.log(results)
    const tests = results.runs[0].tests
    const fails = tests
      .filter(test => test.state === 'failed')
      .map(test => test.title.join(' - '))     // concat suite and test titles
    fs.writeFileSync('failed-tests.txt', fails)
  })
  .catch((err) => {
    console.error(err)
  })

Deleting tests automatically is like playing with a loaded gun.

Better, once you have the list you can prevent failures from running again with cypress-select-tests

// cypress/plugins/index.js

const selectTests = require('cypress-select-tests')

const failures = require('./failed-tests.txt')

// return test names you want to run
const pickTests = (filename, foundTests, cypressConfig) => {
  // found tests will be names of the tests found in "filename" spec
  // it is a list of names, each name an Array of strings
  // ['suite 1', 'suite 2', ..., 'test name']

  return foundTests.filter(fullTestName => {
    return !failures.includes(fullTestName)
  })
}

module.exports = (on, config) => {
  on('file:preprocessor', selectTests(config, pickTests))
}