Basics of Automated Testing - Documentation for Bmc Ami Devx Mainframe Devops
This Section Describes the Fundamentals of Automated Testing. Unit, Functional, and Integration testingTotal Test Supports Developers to Create, Execute and...
This section describes the fundamentals of automated testing.
Unit, Functional, and Integration testing
Total Test supports developers to create, execute and automate unit, functional and integration tests. To make it very simple and understandable, from a technical point of view, the difference between unit and functional test is that a unit test virtualizes all external calls while a functional test runs live.
- Unit test – executing a test case of a program in isolation where all external calls (Files, Db2, sub programs CICS/IMS APIs) are virtualized / stubbed out by the test case.
- Functional test – executing a test case of a program in a live environment with live data.
- Integration test – executing a test case that tests collaboration between two or more programs in a live environment with live data.
The test types are illustrated in the pyramid below. The pyramid illustrates that unit tests should be the major part of test cases. On top of unit tests we then have functional tests and integration tests, and in the end, we have the User Acceptance Testing (UAT), which should be a minor part compared to unit, functional and integration testing.
There are no officially agreed upon clear definitions of different test types and each vendor and customer seems to have their own definitions. As a start let us define what each test type means in relation to Total Test and elaborate a bit more on the differences compared to the technical difference above.
Type | Description |
|---|---|
| Unit Test | This tests the smallest piece of executable code in an application. For COBOL this is a program. It is created by a developer that understands the program code and logic and knows how to, for example, get into a specific IF statement to test this part of the code. The developer is in control and can stub out all external calls from the program to other systems such as IO, Db2, sub programs and CICS/IMS. The developer can decide what an external call from the program should return in a specific test case. A unit test can be described as a white box test where the developer has deep knowledge about the source code, and the test is executed in an isolated environment without requirements for access to external systems and data. |
| Functional Test | A functional test tests a function in an application. A function is typically implemented as a COBOL program. The functional test tests that the program implements its specification correctly and is typically created by a developer or tester/QA role. The tester understands what the program is supposed to do and sets up the functional test case by providing input to the program and defining expected output. The functional test is a black box test where the tester does not need to know about the internal source code of the program, and the program is executed in a live environment with live data and live systems. |
| Integration Test | An integration test tests the collaboration between two or more functions/programs and the data they use. This could, for example, be one program creating an entry in a table and another program reading the entry. The integration test will verify that the collaboration between the programs and their data works as expected. It can be made by a tester or developer. Like for functional testing, integration tests are executed in a live environment with live data and systems. |
Must Read
Shifting left and test-driven development
Shifting left means that testing should be done earlier in the development process, whereas test-driven development means that you have test cases for all software requirements.
- DevOps pipelines—automated tools and processes that enable developers and operations professionals to collaborate to build and deploy code to a production environment.
- Repositories—a place to store code, or software projects. A repository hosting service is an organizational management tool which offers a transparent view into the traditionally opaque workflow process of software development and production.
- Test-driven development—a software development process that relies on software requirements which are converted into test cases before the software is completely developed, and tracking software developments by repeatedly testing the software against all test cases.
In DevOps we often talk about shifting left. This means that testing should be done earlier in the development process. Traditionally, testing on the mainframe is done in the later stages of a project, typically as part of end user acceptance testing. This is very late in the process for finding issues. Shifting left means that bugs should be found as early as possible. The earliest they can be found is at the same time that they are introduced into the source code by the developer. This is where unit testing comes in. A developer should write unit tests that test the changed code and in this way the bug is found as soon as it is introduced into the code. This is the fastest way to detect and correct bugs and has the largest benefit for increasing velocity, quality, and efficiency.
Functional test is typically done in a test environment with live data and is performed later in a sprint where a developer has moved the compiled load modules from a development environment to a test environment. This means that functional tests find bugs later, and hence the feedback to development is slower and more costly.
Integration testing typically comes after functional testing which means that it is more expensive to detect bugs compared to both unit and functional testing. The following illustration shows how unit, functional, and integration tests can be used by developers and testers in different environments during an agile sprint.
Elena Rostova holds a Master's degree in Public Health Journalism. She covers groundbreaking medical research, holistic wellness trends, mental health awareness, and nutritional science.
