Test 2 Impact Testing Methodology
As organizations look to software as a source of generating business value, they expect Quality@Speed. They have also started to acknowledge testing as an integral process of ensuring quality throughout the software development lifecycle rather than an isolated process in the end.
A mature and capable testing function can help an organization with:
- Measure the completeness of a software application
- Define low level detailed requirement
- Build knowledge about features and flows of the application
- Elaborate various components of the system and their interactions
- Develop insights on robustness of different parts of the application
- Track and monitor the application and continuously evaluate its state of readiness
- Evolve the application to make it more usable and beneficial for the end users thus driving it’s adoption
At IOO Labs we recognise the potential and challenges of the testing process and testing team and have stitched together a testing methodology “Test2Impact” to help address the issues. At the centre of this methodology is our in-house integrated testing platform “Rover” which attempts to take testing process beyond defect finding and expand its impact to project management, requirement management and application knowledge building & documentation.
The “Test2Impact” methodology focuses on the following business goals:
- Efficiently manage test cases for a project
- Ensure that test cases remain up-to-date with regards to the features
- Achieve binding of the actual steps written in test cases to what gets executed during testing
- Execute test cases in parallel to increase team’s productivity and optimize downtime
- Shorten the test execution time to avoid project delays caused by stretched development time
- Track and monitor project completion status over multiple release cycle in an efficient manner
- Drive high coverage of application scenarios and eliminate redundancy in testing process
These business goals are achieved through a combination of solutions built as part of Rover testing platform and the process and practices built around it.
Rover Testing Platform
The entire solution is divided into three broad focus areas:
- Test Case Management – The test case management focuses on digitization of test cases and reusing them multiple times. This significantly increases efficiency and also helps in documentation
- Reports & Dashboard – The report and dashboard part concerns with verifying completeness of the application, identifying areas of improvement and also charting progress through multiple development iterations
- Testing Operations & Execution – The testing operation & execution part strives to shorten the time taken to complete rounds of testing thus enabling faster shipping of software features
The solution is deployed partially in a central cloud server and partially in local machines. While there is one central server, there can be multiple local machines - pertaining to an automation engineer. The local machines are set up and configured in the cloud server using a “machine administration workflow”.
The test case management, reports and dashboards are implemented in the central cloud server, whereas the testing execution code resides in the multiple local machines. The execution code interacts with the central server to fetch test case related information and data while updating execution level information to the central server.
The solution uses a multi-tenancy approach where multiple client organizations are created with clear and hard separation of organizational data. Within one organization multiple users and projects are added. Also within each project multiple test cases are added. All of these are stored in the central cloud server.
Setup related information such as mobile app information, device information, browser information etc. are captured and configured in the cloud server for later retrieval and usage.
Processes & Practices around Rover
- Creation of Test cases within Rover at the beginning of the development cycle
- Organising of test cases into test suites for better management and efficient execution
- Process to evaluate high coverage and low redundancy in the test cases
- Creation of test data for the respective test cases at the end of development cycle
- Execution of test cases at the end of development cycle
- Defect Tracking and management after each cycle
- Creation of meaningful test reports which gives actionable insights
- Creation of test cases using scientific format
- Ensuring each test case is atomic which means It focuses on verifying only one thing
- Spending more time in-depth analysis and defining test cases compared to execution of test cases to ensure that testing is purposeful and robust
- Focus on creation of meaningful test data to ensure high coverage of various application scenarios
- Root cause analysis of each defect for efficient triaging
- Repeat test execution for multiple browsers or multiple devices
Benefits and Advantages
IOO Labs’ “Rover” platform provides the following key benefits to our clients:
- Reusable automation components help reduce effort and time in automation framework creation
- Creating test cases digitally and reusing them for multiple execution helps in improving efficiency of testing
- Simple and structured definition of test cases makes it comprehensible even for people with non-technical background
- Capturing all testing artifacts, other than test cases, such as test suite, build details, execution details, defect report helps in managing the testing process more efficiently
- Ability to add multiple test data to the same test case helps in performing data driven testing
- Scheduling of test execution enables in reducing the testing down time
- Automation of test case execution process combined with parallel execution in multiple machines shortens the time required to cover all test cases
- Combining mobile and web apps testing along with API testing gives a comprehensive and integrated testing platform for various testing needs
- Separation of test case creation and test case execution provides the flexibility to create specialists focusing on either test case writing or test case automation
- The test result reports help in tracking completeness of features or projects as well as pinpointing the faulty areas of application
All of these lead to the following advantages for clients:
- Bring testing into the forefront of project management to drive application development
- Help manage requirements effectively as the understanding of the application deepens
- Build a knowledge repository of the feature and flow of the application without having to go through the application
- Significantly improve the robustness of application through testing
- Reduce the overall cost involved in testing the application
The process of testing does not end with development and is very critical throughout the software lifecycle. It plays a very different role when the software is in development mode and adds a different set of values when the software is in production mode. At IOO Labs, we endeavour to significantly up level the contribution of testing service in the overall portfolio of software related services, through our comprehensive Test2Impact testing methodology, offerings and philosophy.