TESTING CONCEPTS

Testing

Software testing is a critical element of software quality assurance and represents the ultimate review of specification, design, and coding.

Testing Objectives

Testing is a process of executing a program with intent of finding an error.
A good test case is one that has a high probability of finding an as-yet undiscovered error.
A successful test is one that uncovers an as-yet undiscovered error.

Testing Principles

Before applying methods to design effective test cases, a software engineer must understand the basic principles that guide software testing.

All tests should be traceable to customer requirements
Tests should be planned long before testing begins
Testing should begin “in the small” and progress toward testing “in the large”
Exhaustive testing is not possible
To be most effective, an independent third party should conduct testing.

Testability

Software testability is simply how easily a computer program can be tested. A set of characteristics that lead to testable software are:

Operability: The better it works, the more efficiently it can be tested

Observability: What you see is what you test

Controllability: The better we can control the software; the more the testing can be automated and optimized.

Decomposability: By controlling the scope of testing, we can more quickly isolate problems and perform smarter retesting.

Simplicity: The less there is to test, the more quickly we can test it.

Stability: The fewer the changes, the fewer the disruptions to testing.

Understandability: The more information we have, the smarter we will test

Quality Assurance & Quality Control

Quality assurance consists of the auditing and reporting functions of management. The goal of quality assurance is to provide management with the data necessary to informed about product quality, thereby gaining insight and confidence that product quality is meeting its goal.

Quality control is the series of inspections, reviews, and tests used through the development cycle to ensure that each work product meets the requirements placed upon it. Quality control activities may be fully automated, entirely manual, or a combination of automated tools and human interaction.

Verification & Validation

Verification refers to the set of activities that ensure that software correctly implements a specific function. Validation refers to a different set of activities that ensure that the software that has been built is traceable to customer requirements.

Verification & Validation encompass a wide array of SQA activities that include formal technical reviews, quality and configuration audits, performance monitoring, simulation, feasibility study, documentation review, database review, development testing, qualification testing and installation testing.

Types of Testing

White Box Testing

White box testing sometimes called glass-box testing is a test case design method that uses that the control structure of the procedural design to derive test cases. Using white box testing methods, the software engineer can derive test cases that guarantee that all independent paths within a module have been exercised at least once, exercise all logical decisions on their true and false sides, execute all loops at their boundaries and within their operational bounds.

White Box Testing Techniques:

Basic Path Testing
Flow Graph Notation
Cyclomatic Complexity
Deriving test cases

Block box Testing

Block box testing focuses on the functional requirements of the software. The block box testing enables the software engineer to derive sets of input conditions that will fully exercise all functional requirements for a program. Block box testing attempts to find errors in interface, errors in data structures or external data base access, performance errors, initialization and termination errors.

Block box Testing Techniques:

Equivalence Partitioning
Boundary Value Analysis
Comparison Testing

Software Testing Strategy

Unit Testing

Unit testing makes heavy use of white-box testing techniques, exercising specific paths in a module’s control structure to ensure complete coverage and maximum error detection.

Integration Testing

Integration Testing addresses the issues associated with the dual problems of verification and program construction. Block box test case design techniques are the more prevalent during Integration.

Validation Testing

Validation Testing provides final assurance that software meets all functional, behavioral, and performance requirements. Black-box testing techniques are used exclusively during validation.

Acceptance Testing

A series of Acceptance tests are conducted to enable the customer to validate all requirements. The end user rather than system developer conducts it. An acceptance Test can range from an informal test drive to a planned and systematic executed series of tests.

Alpha and Beta Testing

A customer conducts the Alpha testing at the developer’s site. The developer records errors and usage problems. Alpha tests are conducted in a controlled environment.

The Beta test is conducted at one or more customer site by the end user(s) of the software. Unlike Alpha testing, the developer is generally not present. The customer records all problems that are encountered during beta testing and reports these to the developer at regular interval.

Regression Testing

Re-testing after fixes or modification of the software or its environment. It is very difficult to determine how much re-testing is needed. Automated testing tools can be especially useful for this type of testing.

System Testing

System Testing verifies that all elements mesh properly and that overall system function/performance is achieved. It covers all combined parts of a system.

System testing is actually a series off different tests whole primary purpose is to fully exercise the computer-based system.

Series of Tests in System Testing:

Recovery Testing

Recovery testing is a system test that forces the software to fail in a variety of ways and verifies that recovery that recovery is properly performed.

Stress Testing

Stress Tests are designed to confront programs with abnormal situations. Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume.

Performance Testing

Performance testing occurs throughout all steps in the testing process. Even at Unit level, the performance of an individual module may be assessed as white-box tests are conducted. Performance testing after coupled with stress testing and often require both hardware and software instrumentation.

Test Plan

Test plan is a document, which contains complete plan. It is the first document to be designed which describes the objective, scope, approach and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The complete document will help people outside the test group understand the why and how of product validation.

The Test Plan contains the following:

Objective
Scope
Strategy
Features to be Tested
Features not to be Tested
Pass/Fail Criteria
Exit/Resumption Criteria
Test Deliverables
Test Schedule
Risks
Environment Requirement
Staffing
Release Document

Test Cases

A Test Case is simply a test with formal steps and instructions; Test Cases are valuable because they are repeatable, reproducible under the same environments, and easy to improve upon with feedback. A Test Case is the difference between saying that something seems to be working okay and proving that a set of specific tasks are known to be working correctly.

The general structure of Test case:

Test Case ID:

Project:

Version:

Feature:

Purpose:

Test Data:

Date: Test Plan:

Sno	Description	Expected	Actual	Error	Remarks

Bug Life Cycle:

Bug is nothing but an exception or error. The bugs will have different severities like low (also called cosmetic), medium, High (also called Catastrophic). All GUI errors are come under severity low and all functionality errors come under Medium Severity and all Security bugs come under High severity bugs.

Test Metrics/ Bug Metrics

Test Metrics contains the following

1. Number of Test Cases Designed

2. Number of Test Cases Executed

3. Number of Test Cases Pending

4. Number of Test Cases Failed

Bug Metrics contains the following

1. Total number of Bugs found

2. Number of Fixed Bugs

3. Number of Pending Bugs

Thursday, December 01, 2011