Types of Testing Approaches

Types of Testing Approaches:  

1. White Box Approach: Structural testing approach focuses on application internals. We can call it as Program-based.

Characteristics:

a. Module design

b. Implementation

c. Do modules / functions meet functional and design specifications?

d. Do program structures meet functional and design specifications?

e. How does the program work?

Type of Test during White Box Approach:

a. Unit Testing: To verify the pieces of code against the design, to remove problems from the code, and to ensure that each component of the code executes according to functional and design specifications. Generally, developers do unit testing.

b. Integration Testing: To verify that each software unit interfaces correctly with other software units. Developers, also testers perform integration testing.

Techniques & Tools:

a. Fault insertion – Coding tools

b. String Test – GUI and Server test tools

c. Error handling – Error handling Tools

d. Statement coverage – Static and Dynamic Analyzers

e. Decision coverage - Static and Dynamic Analyzers, Coverage analysis Tools

f. Condition coverage - Static and Dynamic Analyzers, Coverage analysis tools

g. Path coverage - Static and Dynamic Analyzers, Coverage analysis tools

h. Data flow coverage – Data modeling tools, Flow diagram editors

i. Memory leak – Memory usage tools

j. Cyclomatic complexity – Cyclomatic complexity Analyzers

2. Black Box Approach: Functional testing approach focuses on application externals. We can call it as Requirements-based or Specifications-based.

Characteristics:

a. Functionality

a. Requirements, use, standards

b. Correctness

c. Business forms, Documents

d. Does system meet business requirements?

Type of Test during Black Box Approach:

a. System Testing: System test demonstrates that all programs function works together as designed. This test is performed when all the individual component objects for a release have been integrated into a single system. Documented nun-functional requirements such as performance, volume, stress, capacity and security are also verified. Particular attention is paid to regression testing during this phase.

 

b. User Acceptance Testing: The testing performed for or by user is called user acceptance testing or UAT. The purpose is to ensure that end users are satisfied with the functionality and performance of the system.

Techniques & Tools:

a. Equivalence partitioning – Develop program code to perform tests

b. Boundary value analysis – Develop program code to perform tests

c. Cause-effect graphing – Flow graphing tools

d. Random testing – GUI test tools

e. Error Guessing – GUI test tools

f. Regression testing – GUI / Server test tools

g. Stress testing – Load test tools

h. Replication testing - Load test tools

i. Data Integrity testing – Data analysis tools

j. Backup and recoverability testing – Load test tools / Server test tools / GUI test tools

k. Configuration testing – Multi platform test tools

l. Performance Testing – Load test tools

m. Functional testing – Load test tools / GUI test tools / Server test tools

n. Security testing – Security test tools

o. Operational Readiness Testing – Load test tools / server / GUI tools

p. User Acceptance testing – GUI test tools

q. Compatibility / conversion testing – Load / GUI / Server test tools

r. Benchmark Testing – Benchmarking tools

s. Usability testing – Usability measurement tools

t. Alpha / Beta testing – Load / Server / GUI test tools

Equivalence Partitioning: Three basic types of equivalence classes apply when testing for range and domain errors: in-bound, out-of-bound, and on-bound situations. It is good practice to develop test procedures to examine boundary cases.

One advantage with equivalence partitioning is that it reduces the scope of exhaustive testing to a well-defined set of test procedures, as opposed to an ad-hoc definition of test procedures

One disadvantage relates to the fact that the resulting test procedures do not include other types of tests that have a high probability of finding an error.

Cause-effect graphing: It is a technique that provides a concise representation of logical conditions and corresponding actions. Four steps are involved in this technique:

a. Listing of causes and effects

b. A cause-effect graph is developed

c. A graph is converted into decision table

d. Identification of causes and effects by reading functional specifications

Boundary value analysis: Boundary value analysis can be applied to both functional and structural testing levels. Boundaries define three classes of data: good, bad, and on the border. Boundary testing uses values that lie in or on the boundary (such as end points), and maximum / minimum values (such as field lengths).

Functionality Testing: The primary objective of this test is to assess whether the application does what it is supposed to do in accordance with specified requirements.

A functional test exercises a system application with regard to functional requirements with the intent of discovering nonconformance with end user requirements. 

Regression Testing: The whole notion of conducting tests is aimed at finding and documenting defects and tracking them to closure. The test engineer needs to be sure that the action performed to fix the software does not, in turn, create a new error in another area of the software system. Regression system determines whether any errors have been introduced during the error-fixing process.

When using automated test tools, the test scripts recorded previously can be run again to see whether it produces other errors.

Security Testing: Security tests involve checks to verify the proper performance of system access and data access mechanisms.

Stress Testing: These tests are performed when processing of transactions reaches its peak and steady loads of high-volume data are encountered. Stress testing measures the capacity and resiliency of the system on each hardware platform.

In this technique, multiple users exercise specific functions concurrently and some use values outside of the norm. Several types of stress testing exists:

a. Unit stress tests generate stress on single interface.

b. Module stress testing verifies that business functions are processed in a common area

c. System stress testing involves loading the system using high-volume transactions

Performance Testing: Performance tests verify that the system application meets specific performance efficiency objectives. Performance testing can measure and report on such data as Input / Output rates, total number of Input / Output actions, average database query response time, and CPU utilization rates

Usability Testing: Usability tests verify that the system is easy to use and that the user interface appearance is appealing. Test engineer needs to evaluate the application from the prospective of the end user.

Random Testing: Random tests consist of spontaneous tests identified by the test engineer during test development or execution. These types of tests are also referred to as Monkey tests.

Data Integrity Testing: It verifies that data are being stored by the system in a manner where the data is not comprised by updating, restoration, or retrieve processing. Validation to be performed can include checking data fields for alphabetic and numeric characters, for information that is too long, and for correct date format.

Conversion Testing: Conversion testing measures and reports the capability of the software to convert existing application data to new formats. Conversion accuracy is measured by comparing the test data dump with the new database.

Backup and Recoverability Testing: It verifies that the system meets specified backup and recoverability requirements. These tests help to prove that the database and software can recover from partial or full catastrophic failures of system hardware or software.

Configuration testing: Configuration testing verifies that an application operates properly on machines with different hardware and software configurations. Such tests check for compatibility issues.

Operational Readiness Testing: This test technique helps determine whether a system is ready for normal production operations. All valid and invalid values are defined and applied during these tests.

Alpha / Beta Testing: Most software vendors use alpha and beta testing to uncover errors through testing by the end user. Customers at the developer’s site, with the developer present, usually conduct alpha testing.

Beta testing is usually conducted at one or more customer sites with the developers not present. 

Common QA Testing Terms and Definitions:

ATLM: Automated Test Life-Cycle Methodology

CMM: Capability Maturity Model

TMM: Test Maturity Model

AUT: Application under test

UAT: User Acceptance test 

Smoke Testing: The smoke test (build verification test) focuses on test automation of the system components that makeup the most important functionality. Whenever a new software build is received, the test engineer plays back the smoke test, verifying the major functionality of the system still exists.

Benefits of Automated testing:

1. Production of a reliable system.

2. Improvement of the quality of the test effort

3. Reduction of the test effort and minimization of the schedule

Test Script: A test script contains the detailed sequence of manual and automated actions, as well as the setup information to execute a test case.

A test script contains sections for:

• Test setup

• Actions or Procedures to complete

• Expected results

• Actual results

Test conditions coverage matrix: This matrix is work sheet that is used for planning, and for illustrating that all test conditions are covered by one or more test cases.

Test Requirements coverage matrix: This matrix is work sheet that is used to illustrate that all requirements are covered by one or more test cases.