Testing Metrics

Testing Levels

  • 20% - End to End (in a real env)
  • 60% - Integration Testing (mocked env)
  • 20% - Unit

Testing Tenets

  • How confident are you that when your testing suite is green you can release to production w/o quality issues?
    • Optimize for confidence
  • We rely on the fact that external API changes infrqeuently, and API contract is stable (hence we can stub most of the responses and expect no breaking changes there)
  • Integration tests are used for testing integration of application and layer above source of data

Input Metrics

  • Number of scenarios (tests) at each level

Output Metrics

  • Level of confidence that with current test suite we can deploy to production w/o quality issues (do not count in manual smoke test). (0 - 100%)