Test_Procedure_for_Interview

Test Procedure Test for interviewing – Administration and Scoring Instructions

Uses: View All Cells Design.doc

Instructions to Interview Candidate:

v  Create a test procedure from this spec. Include test titles and test procedure sections, but don’t include steps to reproduce. The expected results should be clear in the test titles.

v  Keep the test procedure to two pages

v  Please spend no more than one hour

Scoring the results:

v  Good document format and organization – 10 points

v  Sections of their test procedure – setup, data creation, things that apply to every test (like logs to look at), test procedure history, etc – 20 points

v  Types of tests included: Functional, Negative, User, Boundary, Stress, Upgrade, etc – 30 points

v  They should have questions about incomplete or unclear information – 20 points

v  Number of test cases that are valid tests, did they see a lot of tests? – 20 points

Here’s my outcome for comparison.  Consider this example as roughly 90-100% of a perfect score.  For example, if the candidate finds 24 valid test cases – the score would be 20 points because the example has 22 test cases.

*******************************

Test Procedure

Test Procedure History

Test Procedure Contents

Reference Documents:

Data Requirements:

Need database with some # of dimensions.  Some records will have one or more dimensions, others will have none.

Preparation

Create cube count for multiple dimensions.  Include all dimensions so some counts will be 0.

Start with a clean installation? Consider if this is needed.  Test considers yes.

For all tests, please monitor:

Relevant log files are: Please monitor these during testing for errors or exceptions.

Relevant memory leak: Please monitor task manager for memory leaks

Test Case Outline

Verify GUI

1. Start Product and verify starting GUI
2. Record beginning memory usage
3. Verify no cube is presented in the GUI if there are no counted scenarios
4. Verify that refiner link is enabled for counted scenario
5. Verify that refiner link is enabled for counted scenario with “Include Dimension Value with no Customers” checked
6. Verify that refiner link is enabled for non-counted scenario with “Include Dimension Value with no Customers” checked
7. Verify that refiner link is disabled for un-counted scenario with “Include Dimension Value with no Customers” is unchecked. 
8. Verify GUI message in #6 is helpful
9. Record interim memory usage
10. Save & Exit
11. Reopen product and verify that settings from test case 6 are persisted

Regression

12. Verify scenario created with all dimensions present has correct counts

New Functionality

13. Verify Help is updated to reflect the new functionality
14. With “Include dimension Value with no Customers” selected, Verify scenario created with some dimensions having a count of zero has correct counts.
15. With “Include dimension Value with no Customers” disabled, Verify scenario created with some dimensions having a count of zero has correct counts.
16. Verify that “ShowEmptyValues” flag can be overwritten in Scenario Wizard
17. Verify that “Show Empty Values” flag can be overridden in Template Wizard

Upgrade Behavior

18. ?? Need more information

Performance

19. Verify performance for the cube draw with “Include Dimension Value with no Customers” selected and without.

Stress Test or Boundary Condition

20. Where all dimension values are 0, count with “Include dimension Value with no Customers” selected, verify counts are correct.
21. Where all dimension values are 0, count with “Include dimension Value with no Customers”  NOT selected, verify that the cube behaves (how? Need more information in the specification)

User Scenario

22. Verify that what the counted data can be used by the next process typical in a user scenario.

Cleanup Section

Would we recommend any clean up after this test?

Some questions:

v  The terminology is not consistent in the spec – the “Include Dimension Values with no Customers” flag is called various things.

v  Upgrade behavior is missing from the spec

v  Cube draw behavior when no dimensions have counts is not specified.  Will there be a message to the user?