Software engineers often hate testing. However, it is a vital part of a software development process. Lacking automated tests can create verification issues once software matures.
Through this article, I will guide you through the benefits, costs, and potential risks of test automation. The value depends on the needs and realities of business. This article will not go into specific testing concepts (TDD, UT, mocking, etc). While they are important, I will omit them for simplicity.
Benefits
The major benefits of automated tests are:
- reduced cost of maintenance,
- a reduced price of future development,
- time leverage of QA engineers
Well-structured tests enable developers to modify code in a safe environment, reducing the price of future development. By reducing the length of the feedback loop (time between introducing and discovering a bug), the cost of a bug is reduced. This concept deserves its attention, but for now, let’s assume it’s true.
Time leverage is an important benefit. With automated tests, QA engineers can focus on unstructured testing rather than executing manual test plans. I have witnessed entire teams wasting dozens of man-months just to execute a plan of 8000 P1 tests, once. Had these tests been automated, the entire effort would provide the same results in 4 days, with no human error involved.
Costs
Building a framework on top of 3rd party software is a mandatory cost. It can be broken down into three major categories:
- Cost of initial development
- Cost of maintenance
- Cost of occasional improvements, optimization, or extension
Onboarding cost (through workshops and presentations) varies on the organization’s size and structure, but it always exists.
The cost of automating test cases is the largest individual cost. However, it follows economies of scale – more tests you write, cheaper they become. The main enabler of this is a well-written framework, that reduces the time needed to automate a test from 1 day to 30 minutes. A good rule of thumb is that about 20% of the development effort is used on writing tests.
The cost of maintaining tests is a recursive cost that scales with the number of tests. The older the tests are the less effort is used to maintain them. About 10% of the initial cost is needed for maintenance.
The last, but not the least, the cost of required 3rd party software exists, but should already be paid through various continuous delivery solutions.
Risks
The main risks are related to the quality of the tests themselves. A test that is not well asserted gives a false sense of security, thus increasing the overall cost of maintenance. The quality of the framework can affect both schedule and cost. And finally, in larger ecosystems, delays can happen due to inter-team dependencies. For example, when one team is responsible for the framework, others depend on the team’s priorities.
Summary
The main goal of this article is to explain the advantages of having automated tests from a business perspective. We went through the benefits of using automated tests as a regular development practice, costs this effort will incur, and also addressed the most frequent risks. With this, I hope you can build your case for introducing automated tests into your company or expand your current testing capabilities.
The good starting point is to learn how automated tests fit the standard life-cycle of your software. Sooner you do it, the bigger the benefits are.
Happy testing!