Software Testing

Recap
• Testing methods / Types
– Black Box testing
– White Box testing
– Incremental / Thread testing
• Testing levels Vs testing methods
• Testing Strategy
Contents

• Debugging Process
• Test cases
• What is a test case?
• Writing better test cases
• Examples
Debugging Process

• Debugging occurs as a consequence of successful testing

• It is still very much an art rather than a science
• Good debugging ability may be an innate human trait
• Large variances in debugging ability exist
• The debugging process begins with the execution of a test case
• Results are assessed and the difference between expected and
actual performance is encountered
– This difference is a symptom of an underlying cause that lies hidden
– The debugging process attempts to match symptom with cause, thereby
leading to error correction
Why is Debugging so Difficult?

• The symptom and the cause may be geographically remote

• The symptom may disappear (temporarily) when another error is
corrected
• The symptom may actually be caused by non-errors (e.g., round-off
accuracies)
• The symptom may be caused by human error that is not easily
traced

(continued on next slide)

Why is Debugging so Difficult?
(continued)

• The symptom may be a result of timing problems, rather than

processing problems
• It may be difficult to accurately reproduce input conditions, such as
asynchronous real-time information
• The symptom may be intermittent such as in embedded systems
involving both hardware and software
• The symptom may be due to causes that are distributed across a
number of tasks running on different processes
Debugging Strategies

• Objective of debugging is to find and correct the cause of a software

error
• Bugs are found by a combination of systematic evaluation, intuition,
and luck
• Debugging methods and tools are not a substitute for careful
evaluation based on a complete design model and clear source
code
• There are three main debugging strategies
– Brute force
– Backtracking
– Cause elimination
Strategy #1: Brute Force

• Most commonly used and least efficient method

• Used when all else fails
• Involves the use of memory dumps, run-time traces, and output
statements
• Leads many times to wasted effort and time
Strategy #2: Backtracking

• Can be used successfully in small programs

• The method starts at the location where a symptom has been
uncovered
• The source code is then traced backward (manually) until the
location of the cause is found
• In large programs, the number of potential backward paths may
become unmanageably large
Strategy #3: Cause Elimination

• Involves the use of induction or deduction and introduces the

concept of binary partitioning
– Induction (specific to general): Prove that a specific starting value is
true; then prove the general case is true
– Deduction (general to specific): Show that a specific conclusion follows
from a set of general premises
• Data related to the error occurrence are organized to isolate
potential causes
• A cause hypothesis is devised, and the aforementioned data are
used to prove or disprove the hypothesis
• Alternatively, a list of all possible causes is developed, and tests are
conducted to eliminate each cause
• If initial tests indicate that a particular cause hypothesis shows
promise, data are refined in an attempt to isolate the bug
Test Cases
Test case

• A test case contains:

– A sequence of Steps describing actions to be performed,
– Test data to be used
– An expected response for each action performed.
• Test Cases are written based on Business and
Functional/Technical requirements, use cases and
Technical design documents.
• There can be 1:1 or 1:N or N:1 or N:N relationship
between requirements and Test cases
• The level of details specified in test cases will vary
depending on Organizations, Projects and also on the
Test Case Template used OR on the Test Management
tool being used in the project.
Test case (cont…)

• Construction of Test Cases also helps in:

– Finding issues or gaps in the requirements
– Technical design itself.
• As Test Case construction activity would make
tester to think through different possible Positive
and Negative scenarios.
• It is test cases against which tester will verify the
application is working as expected.
• Number of test cases to be created depends on
the size, complexity and type of testing being
performed
Test case examples
• Test case for ATM
– TC 1 :- succesful card insertion.
– TC 2 :- unsuccessful operation due to wrong angle card insertion.
– TC 3:- unsuccesssful operation due to invalid account card.
– TC 4:- successful entry of pin number.
– TC 5:- unsuccessful operation due to wrong pin number entered 3 times.
– TC 6:- successful selection of language.
– TC 7:- successful selection of account type.
– TC 8:- unsuccessful operation due to wrong account type selected w/r to that inserted card.
– TC 9:- successful selection of withdrawl option.
– TC 10:- successful selection of amount.
– TC 11:- unsuccessful operation due to wrong denominations.
– TC 12:- successful withdrawl operation.
– TC 13:- unsuccessful withdrawl operation due to amount greater than possible balance.
– TC 14:- unsucceful due to lack of amount in ATM.
– TC 15:- un due to amount greater than the day limit.
– TC 16:- un due to server down.
– TC 17:- un due to click cancel after insert card.
– TC 18:- un due to click cancel after insert card and pin no.
– TC 19:- un due to click cancel after language selection,account type selection,withdrawl
selection, enter amount
• Test cases for a web page
– Testing without entering any username and password
– Test it only with Username
– Test it only with password.
– User name with wrong password
– Password with wrong user name
– Right username and right password
– Cancel, after entering username and password.
– Enter long username and password that exceeds the set limit of
characters.
– Try copy/paste in the password text box.
– After successful sign-out, try “Back” option from your browser.
Check whether it gets you to the “signed-in” page.
Some terminologies…
• Documented Test Cases are referred to as
Test Scripts.
• Related test cases or test scripts are
collectively referred to as Test Suite.
Writing better test cases
1. Requirement ID(s) being covered in the Test Case.
2. Test Condition(s) and Expected Result(s) being exercised in the Test Case.
3. Initial setup required for executing the test script. This could be
environment or data or configuration setup to be done before running the
test case.
4. Post execution activities. For e.g.:- Delete the application user
‘WebAdmin’ after test execution is completed.
5. Priority (High, Medium and Low) of the Test Case. Priority will help the
tester to decide which test case(s) have to be run earlier than others.
6. Complexity of the Test Case. It will help to identify and filter Test Cases
based on complexity. This would help in assigning test cases to testers,
before test execution.
7. Approximate time required for executing the test case. This entry is
required from Project management perspective to track the productivity and
also to ensure we can still meet the test execution deadlines.
8. Test Steps. This contains instruction on what actions to perform and what
test data to use.
Writing better test cases (cont…)
9. Expected results. Each Test Step will have a corresponding Expected
result field that would specify the expected response.
10. Actual result. Each Test Step will have a corresponding Actual result field.
Tester would enter the details on the response he saw after executing the
test step.
11. Test Step result. Typically this field would contain values Not Applicable,
No Run, Passed, Failed or in progress etc
12. Test Case Version number
13. Test case creation timestamp.
14. Revision history. When and who wrote or modified the test case etc.
15. Test Case status (Draft, completed, reviewed, Not Valid etc.)
16. Test Case execution timestamps.
17. Associated Defects. This field will help to identify what are the existing
defect(s) that are associated with the test case.
18. Project Name
19. Application Name
How to write test cases?
• Requirements
• Use cases
• Design documents
Understanding requirements
• First understand Business requirements get the big picture of what
is being achieved from business perspective.
• Read and understand how Business requirements are exploded into
functional requirements.
• Go through Technical specification and UI design document to gain
understanding of how the business requirements get translated into
application functionality.
• Once you do this exercise obviously you will get at least few
questions or you may even discover few gaps in the requirements.
• Reach out to Business Analyst and Business User (as required) and
get your questions answered. Now you are ready to go ahead with
your Test Scenario identification.
• Identify Test Scenarios, Test Conditions and Expected results. Once
you have sign-off on Test Scenarios, Test Conditions and Expected
results, start creating Test Cases.
Design documents
• Also consider verifying the technical aspects
mentioned in the Technical and look and feel
aspects mentioned in UI design documents.
• Some of the technical aspects like:
– Field Length
– Validation
– Data structures will be documented in-detail in
Technical specification.
• So, during test case construction, one has to
incorporate additional validations as well.
How to write test cases?
• Get the test cases reviewed
• Detail of each test case
• Write test cases in plain English
• Requirement traceability matrix (RTM)
• Provide test data within the test cases
• Test case version control
• Prioritize test cases
• Pre-requisites and clean up sections
unique-test-case-id: Test Case Title

Purpose Short sentence or two about the aspect of the system is being tested. If this gets too long, break
the test case up or put more information into the feature descriptions.

Prereq Assumptions that must be met before the test case can be run. E.g., "logged in", "guest login
allowed", "user testuser exists".

Test Data List of variables and their possible values used in the test case. You can list specific values or
describe value ranges. The test case should be performed once for each combination of values.
These values are written in set notation, one per line. E.g.: loginID = {Valid loginID, invalid
loginID, valid email, invalid email, empty}
password = {valid, invalid, empty}
Steps Steps to carry out the test. See step formating rules below.
1.visit LoginPage
2.enter userID
3.enter password
4.click login
5.see the terms of use page
6.click agree radio button at page bottom
7.click submit button
8.see PersonalPage
9.verify that welcome message is correct username

Notes and NOTE

Questions: QUESTION
Test case review!
• Get your Test Cases reviewed by Business
Analyst and Business Users.
• It is always good to get inputs from Subject
Matter Experts
• It is better to get review comments from
Business Analysts and Business Users before
testing starts rather than they pointing out at the
deficiencies of your test cases during
acceptance testing or Post implementation.
How detailed should be a test
case?
• Test Cases steps should be as detailed as
possible and should not be written at high level.
• Writing detailed test steps is very important
considering the fact that the same person who
wrote the test cases may not always execute the
same test cases.
• If the test cases are not very detailed then the
person who executes the test case for the first
time will not be able to validate the system
thoroughly as he/she might have not gone
through the requirements and will test only as
per the test case steps.
Example of detail
• Example of a Test Case that is written at high level.
– Step No: 1
Step Description: Login to test application with valid user id/password
Expected Result: Home page is displayed
– Step No: 2
Step Description: Click “Logout” link on home page
Expected Result: Login page is displayed
• Example of a Test case written that is written in detail.
– Step No: 1
Step Description: Open URL https://www.example.com/login.asp
Expected Result: Login page is displayed and contain the below fields
a) “User Name” text field
b) “Password” text field
c) “Submit” button
– Step No: 2
Step Description: Once Login page is displayed. Enter valid userid/password.
a) Enter “user1” in “User Name” text field
b) Enter “abc123″ in “password” text field
Expected Result: a) Verify “User Name” is populated with text “user1”
b) Verify text entered in “password” field is masked and is not readable.
– Step No: 3
Step Description: Click “Submit” button
Expected Result: Verify application “Home” page is displayed
a) Verify “Home” page displays “Welcome user1” message on top of left navigation Menu.
b) Verify Left Navigation menu contains links “Directory”, “Submission”, “Latest Links”, “Approve Links”,
“Logout”.
– Step No: 4
Step Description: Click “Logout” link on the left menu.
Expected Result: Verify user is successfully logged out of the application and application login page is
displayed https://www.example.com/login.asp
Why do we write test cases?
• Test case creation could have two broad goals:
1. Test Cases are supposed part of the deliverable to the customer. TC goal credibility in this
case. Typically UAT (acceptance) level.
2. Test Cases are for team internal use only. Typically System level testing. Testing
efficiency should be the goal in this case. The idea is to write test cases based on design
while code is incomplete, so that we could test product quickly once the code is ready.
• In case of agile development goal number one is not applicable
a) TC are used internally, but the goal is credibility, not efficiency. It also means that TC are
dramatically reworked during test execution
Summary