PERFORMANCE DOCUMENTS

        

1. Performance Testing:  An Introduction

1.1   What is Performance Testing?   

           Performance testing is a process to identify the application behavior under controlled conditions (Eg: No.of users, Duration). Performance testing focuses on the volume, load, scalability and stability of the underlying software architecture and hardware in real-time environments.

             Performance testing is aimed at verifying the system’s performance requirements like

Response time, transactional throughput and number of concurrent users. Performance

testing is used to accurately measure the end-to end performance of a system prior to go live.

 1.2 Why is Performance Testing?

                Issues in General:

1.    Poor Response times: The page downloaded my take more than 2 minutes where as it should download with in 10seconds.

2.    Scalability Issues: If the increasing number of users applied on the systems, after it reaches its maximum handling capacity, system starts giving response times from 5 seconds to 50 seconds.

3.    Availability: If the servers are getting shutdown frequently, this impacts the availability of systems.

4.    High Resource Usage: If the application server CPU utilization exceeds 90% which needs to host more number of users than existing.

·         Prevents costly failures of mission-critical applications.

·         Assures performance and functionality under real-world conditions.

·         Locates potential problems before your customers do.

·         Reduces development time & infrastructure costs.

1.3    Objectives of Performance testing:

                   Below are the objectives that are satisfied:

·         Application’s response for the intended number of users.

·         Application’s maximum load resisting capacity.

·         Application’s capacity of handling the number of transactions required by the Business.

·         Application’s stability under expected and unexpected user load.

·         Hardware Capacity Planning.

·         Benchmarking.

·         To identify memory leaks and Application bottlenecks in performance front.

1.4    When to do performance Testing:

    

           1. Capacity Assessment for a new application

           2. Before application goes into production

           3. For Functional changes or enhancements

           4. Performance testing at each level early.

           5. Migration/upgrading

           6. To identify application behavior and performance issues.

1.5 How to do Performance Testing (Flavours):

1.    Load Testing: It is conducted to verify whether the application can meet expected SLAs in terms of Response times, Throughput and Resource utilization e.t.c under different loads of Vusers.

2.    Stress Testing : This performance testing is done to identify the application maximum

handling capacity of users, by determining the breakpoint.

3.    Spike Testing : It is a performance test conducted to validate performance

Characteristics when subjected to load volumes repeatedly increase beyond anticipated production operations for short periods of time.

4.    Endurance/Soak Testing: It is conducted to verify application memory leaks and code inefficiencies. This test is performed for long durations like 5 hours, 10 hours, 1 day…

5.    Volume Testing: These tests are more appropriate for messaging, Batch processing type situations. Here we concentrate more about throughput instead Response time.

2.  Performance testing Life Cycle:

Performance testing life cycle explained:

The above diagram depicts different phases of performance testing and the documents involved in those phases. The Templates of the phases are provided separately.

1.       Requirements Analysis:

                        

       This is the phase where business and technical requirements are identified and gathered. Here the team sends the Requirements gathering questionnaire to the client for asking test requirements, H/w and S/w requirements.

      Following are the some of the requirements gathered:

       1. SLAs (Service level Agreements).

       2. Workload

       3. Volume of data

       4. H/w

       5. Application Architecture and components

       6. Test tool details 

       7. Test types

      8. Test Duration    

  Here a document is provided as a template for this.

2.       Test plan/Design :

    In this phase the test is planned and designed. Planning is mostly about the Test environment setup as per application, work load profile and H/W.

    Test design is mostly about types of tests to be conducted, Number of Users, schedule, Execution plan, Scripts to recorded, Data, Scripts enhancements, metrics to be measured.

  In Most of the organizations they follow a single document as Test plan/design. Below mentioned are the sections to be included.

1                                          INTRODUCTION   

2                                          APPLICATION OVERVIEW

3                                          PERFORMANCE TEST GOALS AND OBJECTIVES       

4                                          PERFORMANCE TEST APPROACH

5                                          IN SCOPE AND OUT Of Scope

6                                          TESTING PROCEDURE (ENTRY& Exit CRITERIA, TRANSACTION TRAVERSAL DETAILS, TEST DATA REQUIREMENTS, Schedule, Deliverables, Workload Criteria)

7                                          TEST ENVIRONMENT (Tool, ENVIRONMENT SETUP, Test Environment Setup)

8                                          COMMUNICATION & REPORTING

9                                          TEST METRICS (Client, Resource, Server)

10                                       ROLES AND RESPONSIBILITIES       

11                                       RISKS & MITIGATIONS      

12                                       ABBREVIATIONS  

13                                       TEST PLAN APPROVAL/ SIGNOFF

  Here a document is provided as a template for this.

3.      Test Development:

  Before going for performance testing, the test environment (S/W & H/W) should be ready and the scripts should be developed. The test scripts are developed based on the protocol and base scripts are scripted.

 Once base scripts are captured, they should be enhanced by performing Correlation (For dynamic value handling), Parameterization (Substitution of values), function and custom functions to meet the requirement.

Once they are enhanced they should be validated against 1 user (Perform single user run) and reviewed and should be base lined. The base lined scripts should be used for test execution.

The output of this phase should be scripts and the performance test isolated environment.

4.       Test Execution:

This is the phase where test executions occur as per the mentioned in the plan for meeting client requirements. Here the sequence of steps to follow:

1.       Decide the test (Load/stress/endurance) and number of users.

2.       Ramp up, duration and Ramp down pattern.

3.       Workload profile, run time settings and upload scripts.

4.       Apply the details and set the monitors to measure the metrics.

5.       Start run and perform online monitoring.

Outputs of this phase are Test results and Test logs.

5. Test Results Analysis:

     During this phase the test results and logs are analyzed for identifying bottlenecks and performance issues. Results are compared with SLAs for all types of metrics and verify whether results met SLAs or not.

If results are not met SLAs, raise performance defects and analysis what is causing that problem and report.

Preliminary test report is prepared shared across. A template is provided here as attachment. For the reported performance bottlenecks /issues engineering team acts and tunes the application and reengineers it wherever required.

For the tuned application Step 4 Test execution is performed until application meets expected SLAs.

6. Test Reporting:

For the analyzed results an executive summary report is prepared with the following and sent across respective stake holders.

1.       Client metrics (Response times, throughput, hits/sec, pages/sec e.t.c) statistics and graphs.

2.       Server and resource metrics statistics and graphs (RAM Disk, Processor, NW and App, Web, DB server metrics).

3.       Performed tests

4.       Test results and report paths,

5.       Suggestions and Recommendations.

6.       Test logs of various tests

7.       Comparisons of results for various tests.

A sample Performance Test execution report is placed here.

3. Test Criteria for Performance testing:

Performance test criteria define the Entry and Exit criteria for Performance testing.

     3.1 Entry Criteria:

Ø  Before Production

Ø  Need to have your performance test requirements and plan should be ready.

Ø  Isolated environment should be ready for performance testing.

  3.2 Exit Criteria:

Ø  Schedule/Phases completed.

Ø  Test summary report and Test closure report should be prepared.

Ø  Bottlenecks and Defects should be tuned and fixed

Ø  All requirements should be within SLAs.

4. Roles and Responsibilities of a Performance Tester:

    

v  Review performance test requirements and assess what performance testing strategy to implement.

v  Able to understand requirements properly to make it conceptual to quantify.

v  Write test plans, test case scenarios, identify load models and metrics to be collected.

v  Ability to identify which protocol is best to use for building scripts and connecting to test environment interfaces.

v  Ability to develop performance test scripts using ANSI C coding.

v  Review the Performance test scripts, plans and scenarios.

v  Review test environment to verify that the required components are available for a performance test run.

v  Knowledge of different types of Performance Test runs and when to apply them to gain useful information on system usage patterns.

v  Experience in conducting various types of performance tests and collecting necessary metrics.

v  Collect and organize Performance Test metrics in graph forms.

v  Write Performance Test analysis documents highlighting key findings and recommendations.

v  Execute Performance Tests and troubleshoot issues found as needed.

v  Prepare preliminary and Executive summary reports.

v  Report status to test management and project team stakeholders.

v  Adhere to already established Performance testing standards.

5. Work load Profiling:

   This defines how the load should be distributed across various scenarios during normal and peak loads.