Driven to automate
- Pressure to improve the customer experience and reduce lead times. Customer demand is driving financial institutions to update software more often or release new products more rapidly. With each new release, software functionality must be tested quickly and manual testing cannot adequately address testing requirements. Automated functional tests are faster, more accurate, and can be performed repeatedly and frequently.
- Risk aversion to post-production defects. When software fails, the cost can be substantial in terms of lost sales, unhappy customers, and time wasted due to duplicated efforts, among other negative outcomes. The later in the development cycle defects are found, the more expensive they are to fix. Studies show that the formal testing phase of most development efforts detects only about half of all defects. Automated functional testing can be used to identify errors earlier in the development cycle.
- Evolving regulatory requirements. The modern complex regulatory environment requires financial institutions to have the flexibility to make application and system changes that organizations don’t always have the time to fully develop and test. In response, IT organizations are implementing Agile development principles, which automated functional testing is better suited to than manual testing.
A defect identified after software has gone live can cost 100 times more to fix than if it were found in earlier stages.
Preventable pitfalls on the road to automation
We see a growing adoption of automated functional testing by financial institutions as these organizations recognize the need to improve how they deliver and update software-based products and services in a competitive, fast-paced environment. However, financial institutions face four key issues as they strive to implement automated functional testing:
- Lack of long-term planning and a roadmap. Financial institutions frequently implement automated testing on an ad hoc basis, without the long-term plan and roadmap required to succeed on a larger scale. Despite initial senior level support and substantial budgets, without a long-term plan for automating functional tests, momentum stalls after the initial project ends.
- Misconception that automated testing is a quick way to cut costs. There’s a widespread belief that automating functional testing will lead to immediate cost reductions because fewer resources are required to perform testing. Automation can eventually generate sustainable cost savings, but in our view, cost alone should not be the main factor when considering whether to automate testing. The benefits of automation far outweigh the drawbacks of manual testing—including potentially substantial financial losses and damage to the organization’s reputation resulting from defective software released to the market.
- Project vs. process. Another common misconception is that automated functional testing is a “once and done” exercise. In reality, it should be considered an ongoing process. For example, test scripts should be maintained as features are upgraded or rolled out to new platforms.
- Fragmented approach to implementation. Many financial institutions have fragmented approaches to implementing automated functional testing, including outsourcing testing to different third party vendors, resulting in multiple contracts, service-level agreements, and pricing structures. Additionally, each vendor may use different tools and implementation frameworks. This fragmented approach leads to duplicated efforts, decreased efficiency and effectiveness, and increased costs.
These challenges help explain why few financial institutions have come close to reaching the industry target of automating 70% of application testing.
Boosting the odds of success: our approach to automation
A common misconception is that automated functional testing should be implemented as a project. In fact, to derive the most benefit from automation, it should be an ongoing process that is embedded into the operations of a business. Our three step approach serves as a guide for financial institutions to implement automated functional testing into their operations.
Step 1: Develop a long-term plan and a roadmap.
When developing a long-term plan and a roadmap, financial institutions need to identify and prioritize test candidates, which should include frequently updated applications and any customer-facing apps. Once identified, financial institutions need to decide whether automated testing will be implemented in-house or outsourced. Following this testing decision, organizations need to develop a communication plan as effective communication with business and IT stakeholders is essential to success.
Step 2: Develop a testing framework and plan.
To develop a testing infrastructure, financial institutions first need to create a test team consisting of individuals with the knowledge and expertise to develop, update and automate test scripts. Once the team is in place, the organization should establish collaboration between testers and developers, which will serve to support a rapid software development approach. With the plan in place, organizations need to remember to start off small with implementing automated functional testing. After small wins, organizations can continue to add automated functional testing candidates until the ratio of manual to automated testing is approximately 30:70. Finally, throughout the implementation of automated functional tests, ad hoc manual tests should also be performed to confirm that automated tests scripts are working property.
Step 3: Centralize management of service providers, tools, and resources.
To avoid the fragmentation that leads to inefficient, ineffective, and costly testing infrastructure, financial institutions should consolidate the number of third-party vendors providing support to automated testing; they should consolidate / rationalize the number of testing tools utilized in automated testing; and they should centralize testing program management.