Automated Test Lifecycle Methodology

The Automated Test Lifecycle Methodology (ATLM) is a testing process and method in which test automation runs parallel to the software development lifecycle.

Automated Test Lifecycle Methodology (ATLM )

By using the systematic approach outlined within the ATLM, IDT is able to organize and execute test activities in such a way as to maximize test coverage within the limits of testing resources. This structured test methodology involves a multi-stage process, supporting the detailed, and inter-related activities that are required to introduce and utilize an automated test tool; develop test design; develop and execute test cases; develop and manage test data and the test environment; as well as document, track and obtain closure on issue/trouble reports.

The emphasis on automated testing represents a paradigm change from manual testing. This change does not simply involve the application of tools and the performance of test automation. Rather, it is a process that is pervasive across the entire test life cycle and the system development life cycle. The ATLM implementation takes place in parallel with the system development lifecycle. For software professionals to successfully make the leap to automated testing, structured approaches to testing must be embraced. The ATLM promulgates a structured, building-block approach to the entire test life cycle, which enables software test professionals to approach software testing in a methodical and repeatable fashion.

The ATLM can be applied to various software development methodologies, whether Waterfall, Agile, Scrum, etc, but ideally to a software development methodology that focuses on minimizing the development schedule while providing frequent, incremental software builds is ideal. The objective of this incremental and iterative development is to engage the user and the test team early throughout design and development of each build so as to refine the software, thereby ensuring that it more closely reflects the needs and preferences of the user and thus addressing the riskiest aspects of development in early builds.

In this environment of continual changes and additions to the software through each software build, software testing takes on an iterative nature itself. Each new build is accompanied by a considerable number of new tests as well as rework to existing test scripts, just as there is rework on previously released software modules. Given the continual changes and additions to software applications, especially web applications, automated software testing becomes an important control mechanism to ensure accuracy and stability of the software through each build. Automating an ever changing code base can be challenging however if automation is done strategically, the biggest challenges can be overcome.

The ATLM, invoked to support test efforts involving automated test tools, incorporates a multi-stage process. The methodology supports the detailed and inter-related activities that are required to decide whether to acquire an automated testing tool, i.e. help with the go-no-go decision. The methodology includes the process of how to introduce and utilize an automated test tool, covers test development and test design, and addresses test execution and management. The methodology also supports the development and management of test data and the test environment, and addresses test documentation to include problem reports.

The Automated Test Life-cycle Methodology (ATLM) is comprised of six primary components:

1. Checklist/Decision go / no go – when and what to automate

i. During Unit Testing
ii. During System Testing
iii. During Land-based testing
iv. Etc.

2. Evaluating estimated Return on Investment (ROI)
3. Managing expectations
4. Value of Test Automation
5. Entrance Criteria

1. Knowing which tools are available, and compatible with your technology, understanding tool needs, and then selecting and buying the compatible test tool
2. List of types of tools available, tool evaluation and recommendations

1. What is involved in introducing test automation
2. Things to consider and Lessons Learned
3. Demos and Training considerations

1. Test Automation Standards – Design and Development Standards
2. Test Automation is software development – requirements/design/develop/test your automation
3. Test Automation Frameworks

1. Maintaining the automated tests
2. If new defects are uncovered, add test scenarios to the automation framework
3. Deliverables from various testing phases

i. Unit Testing
ii. System Testing
iii. Land-based Testing

4. Metrics

i. Measuring Automation success – i.e. number of defects found via automated test scripts, increased testing coverage; etc.
ii. Measuring Test Coverage and completeness
iii. Actual ROI

5. Exit Criteria

1. Evaluate outcome – constant evaluation
2. Evaluate improvement opportunities

Each of these six primary processes are further composed of subordinate processes that are described in the following section.