A Tutorial on

Software Testing

Mr. Mahesh Pore

Lecturer, Computer Engineering
Institute of Civil and Rural Engineering Gargoti, Kolhapur
Levels Of Testing
And Special Tests
Levels Of Testing
Unit Testing
 Unit Testing is a level of software testing where individual
units/ components of a software are tested. The purpose is
to validate that each unit of the software performs as
designed.
 Unit Testing is the first level of testing and is performed
prior to Integration Testing.
 A unit is the smallest testable part of software. It usually
has one or a few inputs and usually a single output.
 It is executed by the Developer.
 Unit Testing is performed by using the White Box Testing
method
 Example: - A function, method, Loop or statement in
program is working fine.
Drivers
 Drivers are used in bottom-up integration testing approach.
 It can simulate the behavior of upper-level module that is not
integrated yet.
 Drivers modules act as the temporary replacement of module
and act as the actual products.
 Drivers are also used for interact with external system and
usually complex than stubs.
 Driver: Calls the Module to be tested.

Now suppose you have modules B and C ready but module A which
calls functions from module B and C is not ready so developer will
write a dummy piece of code for module A which will return values to
module B and C. This dummy piece of code is known as driver.
Stubs
 Stubs are used in top down integration testing.
 It can simulate the behavior of lower-level module that are not
integrated.
 They are act as a temporary replacement of module and provide
same output as actual product.
 When needs to intact with external system then also stubs are
used.
 Stub: Is called by the Module under Test.
Assume you have 3 modules, Module A, Module B and module C.
Module A is ready and we need to test it, but module A calls
functions from Module B and C which are not ready, so developer
will write a dummy module which simulates B and C and returns
values to module A. This dummy module code is known as stub.
Stub Driver
Type Dummy codes Dummy codes

Routines that don’t actually Routines that don’t actually

do anything except declare do anything except declare
themselves and the themselves and the
Description parameters they accept. The parameters they accept. The
rest of the code can then take rest of the code can then take
these parameters and use these parameters and use
them as inputs them as inputs

Used in Top Down Integration Bottom Up Integration

To allow testing of the upper To allow testing of the lower

levels of the code, when the levels of the code, when the
Purpose
lower levels of the code are upper levels of the code are
not yet developed. not yet developed.
Benefits
 Unit testing increases confidence in changing/
maintaining code. If good unit tests are written and if
they are run every time any code is changed, we will
be able to promptly catch any defects introduced due
to the change.
 Codes are more reusable.
 Development is faster.
 The cost of fixing a defect detected during unit testing
is lesser in comparison to that of defects detected at
higher levels.
 Debugging is easy.
Integration Testing
 Integration Testing is a level of software testing where individual units
are combined and tested as a group.
 In integration Testing, individual software modules are integrated
logically and tested as a group.
 Integration testing tests integration or interfaces between
components, interactions to different parts of the system such as an
operating system, file system and hardware or interfaces between
systems.
 As displayed in the image below when two different modules ‘Module A’
and ‘Module B’ are integrated then the integration testing is done.
Integration Testing Approaches
Big Bang integration testing
 In Big Bang integration testing all components or modules are
integrated simultaneously, after which everything is tested as a whole.
As per the below image all the modules from ‘Module 1’ to ‘Module 6’
are integrated simultaneously then the testing is carried out.
Pros And Cons
 Convenient for small systems.

 Fault Localization is difficult.

 Since all modules are tested at once, high risk critical modules are not
isolated and tested on priority.
 Since the integration testing can commence only after "all" the modules
are designed, testing team will have less time for execution in the
testing phase.
Incremental Approach:

 In this approach, testing is done by joining two or more modules that

are logically related. Then the other related modules are added and
tested for the proper functioning. Process continues until all of the
modules are joined and tested successfully.
 This process is carried out by using dummy programs called Stubs and
Drivers. Stubs and Drivers do not implement the entire programming
logic of the software module but just simulate data communication with
the calling module.
 Stub: Is called by the Module under Test.
 Driver: Calls the Module to be tested.
Bottom up Integration
 In the bottom up strategy, each module at lower levels is tested with
higher modules until all modules are tested.
 It takes help of Drivers for testing
Advantages:

•Fault localization is easier.

•No time is wasted waiting for all modules to be
developed unlike Big-bang approach

Disadvantages:

•Critical modules (at the top level of software

architecture) which control the flow of application
are tested last and may be prone to defects.
•Early prototype is not possible
Top down Integration:
 In Top to down approach, testing takes place from top to down following
the control flow of the software system.
 Takes help of stubs for testing.
Advantages:

•Fault Localization is easier.

•Possibility to obtain an early
prototype.
•Critical Modules are tested on
priority; major design flaws
could be found and fixed first.

Disadvantages:
•Needs many Stubs.
•Modules at lower level are
tested inadequately.
Unit Testing Integration Testing
Unit testing is a type of testing to check if the Integration testing is a type of testing to check if
small piece of code is doing what it is suppose different pieces of the modules are working
to do. together.
Unit testing checks a single component of an The behavior of integration modules is considered
application. in the Integration testing.
The scope of Unit testing is narrow, it covers
The scope of Integration testing is wide, it covers
the Unit or small piece of code under test.
the whole application under test and it requires
Therefore while writing a unit test shorter
much more effort to put together.
codes are used that target just a single class.
Integration testing is dependent on other outside
Unit tests should have no dependencies on
systems like databases, hardware allocated for
code outside the unit tested.
them etc.
This is first type of testing is to be carried out This type of testing is carried out after Unit
in Software testing life cycle and generally testing and before System testing and executed
executed by developer. by the testing team.
Integration testing is further divided into different
Unit testing is not further sub divided into types as follows:
different types. Top-down Integration, Bottom-Up Integration and
so on.
Unit testing is starts with the module Integration testing is starts with the interface
specification. specification.
The detailed visibility of the code is comes The visibility of the integration structure is comes
under Unit testing. under Integration testing.
Unit testing mainly focus on the testing the Integration testing is to be carried out to discover
functionality of individual units only and does the the issues arise when different modules are
not uncover the issues arises when different interacting with each other to build overall
modules are interacting with each other. system.
The goal of Unit testing is to test the each unit The goal of Integration testing is to test the
separately and ensure that each unit is working combined modules together and ensure that every
System Testing
 The process of testing of an integrated hardware and
software system to verify that the system meets its specified
requirements.
 It is performed when integration testing is completed.
 It is mainly a black box type testing. This testing evaluates
working of the system from a user point of view, with the help
of specification document. It does not require any internal
knowledge of system like design or structure of the code.
 It contains functional and non-functional areas of
application/product.
 System testing is performed in the context of a System
Requirement Specification (SRS) and/or a Functional
Requirement Specifications (FRS). It is the final test to verify
that the product to be delivered meets the specifications
mentioned in the requirement document. It should
investigate both functional and non-functional requirements.
 It mainly focuses on following:
 External interfaces
 complex functionalities
 Security
 Recovery
 Performance
 Operator and user’s smooth interaction with system
 Documentation
 Usability
 Load / Stress
Recovery Testing
 Recovery testing is a type of non-functional testing technique
performed in order to determine how quickly the system can
recover after it has gone through system crash or hardware failure.
 Recovery testing is the forced failure of the software to verify if the
recovery is successful.
 For example: When an application is receiving data from a
network, unplug the connecting cable. After some time, plug the
cable back in and analyze the application’s ability to continue
receiving data from the point at which the network connection was
broken.
 Example: Restart the system while a browser has a definite
number of sessions and check whether the browser is able to
recover all of them or not.
Security Testing
 Security testing is a testing technique to determine if
an information system protects data and maintains
functionality as intended
 It also aims at verifying 6 basic principles as listed
below:
 Confidentiality
 Integrity
 Authentication
 Authorization
 Availability
 Non-repudiation
 Confidentiality

A security measure which protects against the disclosure of

information to parties other than the intended recipient is by
no means the only way of ensuring the security.
 Integrity

Integrity of information refers to protecting information from

being modified by unauthorized parties

 Authentication
This might involve confirming the identity of a person, tracing
the origins of an artifact, ensuring that a product is what its
packaging and labeling claims to be, or assuring that a
computer program is a trusted one
 Authorization
The process of determining that a requester is allowed to receive
a service or perform an operation.
Access control is an example of authorization.

 Availability
Assuring information and communications services will be ready
for use when expected.
Information must be kept available to authorized persons when
they need it.

 Non-repudiation (acknowledgment)
In reference to digital security, non-repudiation means to ensure
that a transferred message has been sent and received by the
parties claiming to have sent and received the message. Non-
repudiation is a way to guarantee that the sender of a message
cannot later deny having sent the message and that the
recipient cannot deny having received the message.
Example :
A Student Management System is
insecure if ‘Admission’ branch can edit the
data of ‘Exam’ branch
An ERP system is not secure if DEO (data
entry operator) can generate ‘Reports’
 An online Shopping Mall has no security if
customer’s Credit Card Detail is not
encrypted
A custom software possess inadequate
security if an SQL query retrieves actual
passwords of its users
Performance Testing
 Performance Testing is a type of testing to ensure software
applications will perform well under their expected workload.
 A software application's performance like its response time,
reliability, resource usage and scalability do matter.
 The goal of Performance Testing is not to find bugs but to
eliminate performance bottlenecks.
 The focus of Performance Testing is checking a software
program's
 Speed - Determines whether the application responds quickly
 Scalability - Determines maximum user load the software
application can handle.
 Stability - Determines if the application is stable under
varying loads
Performance Testing Process
1) Identify your testing environment –
Do proper requirement study & analyzing test goals and its
objectives. Also determine the testing scope along with test
Initiation Checklist. Identify the logical and physical
production architecture for performance testing, identify the
software, hardware and networks configurations
required for kick off the performance testing. Compare the
both test and production environments while identifying the
testing environment. Get resolve the environment-related
concerns if any, analyze that whether additional tools are
required for performance testing. This step also helps to
identify the probable challenges tester may face while
performance testing.
2) Identify the performance acceptance criteria –
Identify the desired performance characteristics of the
application like Response time, Throughput and Resource
utilization.
3)Plan & design performance tests –
Planning and designing performance tests involves identifying
key usage scenarios, determining appropriate variability
across users, identifying and generating test data, and
specifying the metrics to be collected. Ultimately, these items
will provide the foundation for workloads and workload
profiles. The output of this stage is prerequisites for Test
execution are ready, all required resources, tools & test
data are ready.
4) Configuring the test environment –
Prepare with conceptual strategy, available tools, designed
tests along with testing environment before execution. The
output of this stage is configured load-generation
environment and resource-monitoring tools.
5) Implement test design –
According to test planning and design create your performance
tests.
6) Execute the tests –
Collect and analyze the data.
Problem Investigation like bottlenecks (memory, disk, processor,
process, cache, network, etc.) resource usage like (memory, CPU,
network, etc.,)
Generate the Performance analysis reports containing all
performance attributes of the application.
Based on the analysis prepare recommendation report.
Repeat the above test for the new build received from client after
fixing the bugs and implementing the recommendations
7) Analyze Results, Report, and Retest
Consolidate, analyze and share test results.
Based on the test report re-prioritize the test & re-execute the
same. If any specific test result within the specified metric limit &
all results are between the thresholds limits then testing of same
scenario on particular configuration is completed.
Test objectives frequently include the
following:

 Response time. For example, the product catalog

must be displayed in less than 3 seconds.

 Throughput. For example, the system must support

100 transactions per second.

 Resource utilization. A frequently overlooked aspect

is the amount of resources your application is
consuming, in terms of processor, memory, disk input
output
(I/O), and network I/O.
Types of Performance Testing
 Load testing
 Stress testing
Load Testing

 Load Testing is type of performance testing to check system

with constantly increasing the load on the system until the
time load is reaches to its threshold value.
 Here Increasing load means increasing number of concurrent
users, transactions & check the behavior of application under
test.
 It is normally carried out underneath controlled environment
in order to distinguish between two different systems.
 The main purpose of load testing is to monitor the response
time and staying power of application when system is
performing well under heavy load.
 The successfully executed load testing is only if the specified
test cases are executed without any error in allocated time.
Simple examples of load testing:
 Testing printer by sending large job.
 Editing a very large document for testing of word
processor
 Continuously reading and writing data into hard disk.
 Running multiple applications simultaneously on
server.
 Testing of mail server by accessing thousands of
mailboxes
 In case of zero-volume testing & system fed with zero
load.
Stress Testing
 Stress Testing is performance testing type to check the
stability of software when hardware resources are not
sufficient like CPU, memory, disk space etc.
 It is performed to find the upper limit capacity of the system
and also to determine how the system performs if the
current load goes well above the expected maximum.
 Main parameters to focus during Stress testing are
“Response Time” and “Throughput”.
 Stress testing is Negative testing where we load the
software with large number of concurrent users/processes
which cannot be handled by the systems hardware
resources. This testing is also known as Fatigue testing
Usability Testing
 In usability testing basically the testers tests the ease with
which the user interfaces can be used.
 It tests that whether the application or the product built is
user-friendly or not.
 Usability Testing is a black box testing technique.
 Usability testing also reveals whether users feel comfortable
with your application or Web site according to different
parameters – the flow, navigation and layout, speed and
content – especially in comparison to prior or similar
applications.
 Usability Testing tests the following features of the software.
 — How easy it is to use the software.
— How easy it is to learn the software.
Usability testing includes the
following five components:
 Learnability: How easy is it for users to accomplish basic
tasks the first time they encounter the design?
 Efficiency: How fast can experienced users accomplish tasks?
 Memorability: When users return to the design after a period
of not using it, does the user remember enough to use it
effectively the next time, or does the user have to start over
again learning everything?
 Errors: How many errors do users make, how severe are
these errors and how easily can they recover from the errors?
 Satisfaction: How much does the user like using the system?
Benefits of usability testing to
the end user or the customer:
 Better quality software

 Software is easier to use

 Software is more readily accepted by users

 Shortens the learning curve for new users

Compatibility Testing

 Compatibility Testing is a type of Software testing to check whether your

software is capable of running on different hardware, operating systems,
applications , network environments or Mobile devices.
 Compatibility Testing is a type of the Non-functional testing
 Hardware : It checks software to be compatible with different hardware
configurations .
 Operating Systems: It checks your software to be compatible with different
Operating Systems like Windows , Unix , Mac OS etc.
 Software: It checks your developed software to be compatible with other
software's. For example: MS Word application should be compatible with other
software like MS Outlook, MS Excel etc.
 Network: Evaluation of performance of system in a network with varying
parameters such as Bandwidth, Operating speed, Capacity. It also checks
application in different networks with all parameters mentioned earlier.
 Browser: It checks compatibility of your website with different browsers like
Firefox , Google Chrome , Internet Explorer etc.
 Devices : It checks compatibility of your software with different devices like
USB port Devices, Printers and Scanners, Other media devices and Blue tooth.
 Mobile: Checking your software is compatible with mobile platforms like
Android , iOS etc.
 Versions of the software: It is verifying you software application to be
compatible with different versions of software. For instance checking your
Microsoft Word to be compatible with Windows 7, Windows 7 SP1 , Windows 7
SP 2 , Windows 7 SP 3.
Why it is required

● Checking the platform (os, browser) and other

application program or s/w compatible with your s/w.
● What compatibility standards or guidelines should be
followed that define how your s/w interact with other
s/w
● What type of data your s/w use to interact or share
information with other s/w.
Types of Compatibility Testing

1.Forward Compatibility Testing: This type of testing verifies

that the software is compatible with the newer or upcoming
versions, and is thus named as forward compatible.
2.Backward Compatibility Testing: This type of testing helps
to check whether the application designed using the latest
version of an environment also works seamlessly in an older
version. It is the testing performed to check the behavior of the
hardware/software with the older versions of the
The Isolation of Configuration bug
● Testing new s/w product with multiple versions of platform and s/w
application is a huge task
● compatibility testing of windows Server 2008, programmer fix
numerous bug fixes and improve the performance and add many
new features in the code with old features in server 2003, the goal
is to be 100% compatible with them.
Acceptance Testing
 Acceptance Testing is a last level of the software testing where a
system is tested for acceptability.
 The purpose of this test is to evaluate the system’s compliance with
the business requirements and assess whether it is acceptable for
delivery.
 Usually, Black Box Testing method is used in Acceptance Testing.
 Acceptance Testing is performed after System Testing and before
making the system available for actual use.
 The acceptance test cases are executed against the test data or using
an acceptance test script and then the results are compared with the
expected ones.
 The goal of acceptance testing is to establish confidence in the
system.
 Acceptance testing is most often focused on a validation type testing.
Acceptance Criteria
Acceptance criteria are defined on the basis of the following
attributes
 Functional Correctness and Completeness
 Data Integrity
 Data Conversion
 Usability
 Performance
 Timeliness
 Confidentiality and Availability
 Installability and Upgradability
 Scalability
 Documentation
Advantages Of Acceptance Testing
 The functions and features to be tested are known.
 The details of the tests are known and can be measured.
 The tests can be automated, which permits regression
testing.
 The progress of the tests can be measured and
monitored.
 The acceptability criteria are known.
Disadvantages Of Acceptance Testing
 Requires significant resources and planning.
 The tests may be a re-implementation of system tests.
 It may not uncover subjective defects in the software,
since you are only looking for defects you expect to find.
Alpha Testing
 Alpha Testing is a type of testing conducted by a team of highly
skilled testers at development site. Minor design changes can still be
made as a result of alpha testing.
 For Alpha Testing there is a dedicated test team.
 Alpha testing is final testing before the software is released to the
general public. It has two phases:
 In the first phase of alpha testing, the software is tested by in-house
developers. They use either debugger software, or hardware-assisted
debuggers. The goal is to catch bugs quickly.
 In the second phase of alpha testing, the software is handed over to
the software QA staff, for additional testing in an environment that is
similar to the intended use.
Pros And Cons
 Pros Of Alpha Testing
• Helps to uncover bugs that were not found during previous
testing activities
• Better view of product usage and reliability
• Analyze possible risks during and after launch of the product
• Helps to be prepared for future customer support
• Helps to build customer faith on the product
• Maintenance Cost reduction as the bugs are identified and
fixed before Beta / Production launch
• Easy Test Management
 Cons Of Alpha Testing
• Not all the functionality of the product is expected to be tested
• Only Business requirements are scoped
Beta Testing
 Beta Testing is also known as field testing. It takes place
at customer’s site. It sends the system/software to users
who install it and use it under real-world working conditions.
 A beta test is the second phase of software testing in
which a sampling of the intended audience tries the product
out
 The goal of beta testing is to place your application in the
hands of real users outside of your own engineering team to
discover any flaws or issues from the user’s perspective that
you would not want to have in your final, released version of
the application.
 Beta testing can be considered “pre-release testing.
Advantages of beta testing
 You have the opportunity to get your application into the hands of users prior to
releasing it to the general public.
 Users can install, test your application, and send feedback to you during this
beta testing period.
 Your beta testers can discover issues with your application that you may have
not noticed, such as confusing application flow, and even crashes.
 Using the feedback you get from these users, you can fix problems before it is
released to the general public.
 The more issues you fix that solve real user problems, the higher the quality of
your application when you release it to the general public.
 Having a higher-quality application when you release to the general public will
increase customer satisfaction.
 These users, who are early adopters of your application, will generate
excitement about your application.
Alpha Testing Beta Testing
Basic Understanding
First phase of testing in Customer Validation Second phase of testing in Customer Validation

Performed at developer's site - testing environment. Performed in real environment, and hence activities
Hence, the activities can be controlled cannot be controlled

Only functionality, usability are tested. Reliability and Functionality, Usability, Reliability, Security testing are
Security testing are not usually performed in-depth all given equal importance to be performed

White box and / or Black box testing techniques are Only Black box testing techniques are involved
involved

Build released for Alpha Testing is called Alpha Release Build released for Beta Testing is called Beta Release

System Testing is performed before Alpha Testing Alpha Testing is performed before Beta Testing

Issues / Bugs are logged into the identified tool directly Issues / Bugs are collected from real users in the form of
and are fixed by developer at high priority suggestions / feedbacks and are considered as
improvements for future releases.

Helps to identify the different views of product usage as Helps to understand the possible success rate of the
different business streams are involved product based on real user’s feedback / suggestions.
Alpha Testing Beta Testing
Test Goals

To evaluate the quality of the product To evaluate customer satisfaction

To ensure Beta readiness To ensure Release readiness (for Production

launch)

Focus on finding bugs Focus on collecting suggestions / feedback

and evaluate them effectively

Does the product work? Do customers like the product?

When
Usually after System testing phase or when Usually after Alpha Testing and product is
the product is 70% - 90% complete 90% - 95% complete

Features are almost freezed and no scope for Features are freezed and no enhancements
major enhancements accepted

Build should be stable for technical user Build should be stable for real users
Alpha Testing Beta Testing

Test Duration
Many test cycles conducted Only 1 or 2 test cycles conducted

Each test cycle lasts for 1 - 2 weeks Each test cycle lasts for 4 - 6 weeks

Duration also depends on the number of issues Test cycles may increase based on real user's
found and number of new features added feedback / suggestion

Stake Holders
Engineers (in-house developers), Quality Assurance Product Management, Quality Management, and
Team, and Product Management Team User Experience teams

Participants
Technical Experts, Specialized Testers with good End users to whom the product is designed
domain knowledge (new or who were already part of
System Testing phase), Subject Matter Expertise

Customers and / or End Users can participate in Customers also usually participate in Beta Testing
Alpha Testing in some cases
Alpha Testing Beta Testing
Expectations
Acceptable number of bugs that were missed in earlier Major completed product with very less amount of
testing activities bugs and crashes
Incomplete features and documentation Almost completed features and documentation

Entry Criteria
• Alpha Tests designed and reviewed for Business • Beta Tests like what to test and procedures
requirements documented for Product usage
• Traceability matrix should be achieved for all the • No need of Traceability matrix
between alpha tests and requirements • Identified end users and customer team up
• Testing team with knowledge about the domain and • End user environment setup
product • Tool set up should be ready to capture the feedback /
• Environment setup and build for execution suggestions
• Tool set up should be ready for bug logging and test • Alpha Testing should be signed off
management
System testing should be signed-off (ideally)

Exit Criteria
• All the alpha tests should be executed and all the • All the cycles should be completed
cycles should be completed • Critical / Major issues should be fixed and retested
• Critical / Major issues should be fixed and retested • Effective review of feedback provided by participants
• Effective review of feedback provided by participants should be completed
should be completed • Beta Test summary report
• Alpha Test Summary report • Beta Testing should be signed off
• Alpha testing should be signed off
Alpha Testing Beta Testing

Pros
• Helps to uncover bugs that were not found during • Product testing is not controllable and user may test
previous testing activities any available feature in any way - corner areas are
• Better view of product usage and reliability well tested in this case
• Analyze possible risks during and after launch of the • Helps to uncover bugs that were not found during
product previous testing activities (including alpha)
• Helps to be prepared for future customer support • Better view of product usage, reliability, and security
• Helps to build customer faith on the product • Analyze the real user's perspective and opinion on
• Maintenance Cost reduction as the bugs are identified the product
and fixed before Beta / Production launch • Feedback / suggestions from real users helps in
• Easy Test Management improvising the product in future
• Helps to increase customer satisfaction on the
product

Cons
• Not all the functionality of the product is expected to • Scope defined may or may not be followed by
be tested participants
• Only Business requirements are scoped • Documentation is more and time consuming -
required for using bug logging tool (if required), using
tool to collect feedback / suggestion, test procedure
(installation / uninstallation, user guides)
• Not all the participants assure to give quality testing
• Not all the feedback are effective - time taken to
review feedback is high
• Test Management is too difficult
Smoke Testing
 Smoke testing is the surface level testing to certify that build provided by
development to QA is ready to accept for further testing.
 In smoke testing we only checks the major functionality of the software.
 Smoke testing is also known by the name BAT (Build Acceptance Test)
because it establishes the acceptance criteria for QA to accept and reject a
build for further testing. So apart from smoke testing it is also very
important for software people to know about build.
 A build is called as the version of software, typically one that is still in testing
stage.
 Smoke testing is performed by developers before releasing the build to the
testing team and after releasing the build to the testing team it is performed
by testers whether to accept the build for further testing or not.
 If the build clears the Smoke test, then it is accepted by QA for further
testing, however if the build fails the Smoke test, then it’s rejected and QA
reverts back to previously accepted build.
Smoke Testing Example

In a project there are five modules like

 login,
 view user,
 user detail page,
 new user creation and
 task creation etc.
In these five modules, the developer will first perform the
smoke testing by executing all the major functionality of
modules like user is able to login with valid login credentials
or not, after login new user can be created or not, user that is
created is viewed or not etc. It makes it obvious that the
smoke test is always done by developing team before
submitting the build to the testing team.
Advantages of Smoke testing:
It helps to find issues introduced in integration of modules.
It helps to find issues in the early phase of testing.
It helps to get confidence to tester that fixes in the previous
builds not breaking major features (only features exercised by
smoke testing).
Facts about smoke testing:
A smoke test is a subset of the test cases that is typically
representative of the overall test plan.
Smoke tests are good for verifying proper deployment or
other non invasive changes.
They are also useful for verifying a build is ready to send to
test.
Smoke tests are not substitute for actual functional testing.
Sanity Testing
 Sanity Testing is the subset of Regression Testing and it is performed when we
do not have enough time for doing testing.
 Sanity testing is the surface level testing where QA engineer verifies that all
the menus, functions, commands available in the product and project are
working fine.
 Sanity testing is performed after the build has clear the Smoke test and has
been accepted by QA team for further testing, sanity testing checks the major
functionality with finer details.
 Sanity testing is performed when development team needs to know quick
state of the product after they have done changes in the code or there is some
controlled code change in a feature to fix any critical issue, and stringent
release time-frame does not allow complete regression testing.
 Sanity testing will be done mostly after retest (retest will be done after fixing
the bug). We always use Script for Smoke but do not for sanity.
Sanity Testing Example
For Example in a project there are five modules like login page, home page, user
detail page, new user creation, and task creation etc. So we have the bug in login
page like on login page username field accepts the less than six alpha-numeric
characters which are against the requirements as in requirements it is specified
that username should not be below than six characters but as username accepts
the less than six characters it is the bug.

So now the bug is reported by the testing team to the developer team to fix it.
When the developing team fixes the bug and passed it to testing team than the
testing team checks the other modules of the application means checks that fix
bug does not affect the functionality of the other modules but keep one point
always in mind that testing team only checks the extreme functionality of the
modules, do not go deep to test the details because of the short time so this is
the sanity testing.
Smoke Testing Sanity Testing
Smoke Testing is performed to ascertain Sanity Testing is done to check the new
that the critical functionalities of the functionality / bugs have been fixed
program is working fine

The objective of this testing is to verify the The objective of the testing is to verify the
"stability" of the system in order to proceed "rationality" of the system in order to
with more rigorous testing proceed with more rigorous testing

This testing is performed by the developers Sanity testing is usually performed by

or testers testers
Smoke testing is usually documented or Sanity testing is usually not documented
scripted and is unscripted

Smoke testing is a subset of Sanity testing is a subset of Acceptance

Regression Testing testing
Smoke testing exercises the entire system Sanity testing exercises only the particular
from end to end component of the entire system

Smoke testing is like General Health Check Sanity Testing is like specialized health
Up check up
A Tutorial on
Software Testing

Mr. Mahesh Pore

Lecturer, Computer Engineering
Institute of Civil and Rural Engineering Gargoti, Kolhapur
Special Tests:
Regression Testing
GUI Testing
Web Application Testing
Regression Testing
 Regression Testing is defined as a type of software testing to confirm that a
recent program or code change has not adversely affected existing features.
 Regression Testing is nothing but full or partial selection of already executed
test cases which are re-executed to ensure existing functionalities work fine.
 This testing is done to make sure that new code changes should not have side
effects on the existing functionalities. It ensures that old code still works once
the new code changes are done.
 Regression Testing is required when there is a
 Change in requirements and code is modified according to the requirement
 New feature is added to the software
 Defect fixing
 Performance issue fix
Regression Testing Techniques :
Retest All
This is one of the methods for Regression Testing in which all
the tests in the existing test bucket or suite should be re-
executed. This is very expensive as it requires huge time and
resources.
Regression Test Selection
Instead of re-executing the entire test suite, it is better to
select part of test suite to be run
Test cases selected can be categorized as 1) Reusable Test
Cases 2) Obsolete Test Cases.
Re-usable Test cases can be used in succeeding regression
cycles.
Obsolete Test Cases can't be used in succeeding cycles.
Prioritization of Test Cases
Prioritize the test cases depending on business impact, critical
& frequently used functionalities. Selection of test cases
based on priority will greatly reduce the regression test suite.
Selecting test cases for regression testing
It was found from industry data that good number of the defects
reported by customers were due to last minute bug fixes creating side
effects and hence selecting the Test Case for regression testing is an art
and not that easy.

Effective Regression Tests can be done by selecting following

test cases –

Test cases which have frequent defects

Functionalities which are more visible to the users
Test cases which verify core features of the product
Test cases of Functionalities which has undergone more and recent
changes
All Integration Test Cases
All Complex Test Cases
Boundary value test cases
Sample of Successful test cases
Sample of Failure test cases
Regression Testing Tools
If your software undergoes frequent changes, regression testing costs will
escalate.
In such cases, Manual execution of test cases increases test execution time
as well as costs.

Following are most important tools used for both functional and regression
testing:
Selenium: This is an open source tool used for automating web applications.
Selenium can be used for browser based regression testing.

Quick Test Professional (QTP): HP Quick Test Professional is automated

software designed to automate functional and regression test cases. It uses
VBScript language for automation. It is a Data driven, Keyword based tool.

Rational Functional Tester (RFT): IBM's rational functional tester is a Java

tool used to automate the test cases of software applications. This is
primarily used for automating regression test cases and it also integrates
with Rational Test Manager.
GUI Testing

 There are two types of interfaces for a computer application.

 Command Line Interface is where you type text and computer responds to
that command.
 GUI stands for Graphical User Interface where you interact with the
computer using images rather than text.
 GUI testing is the process of testing the system's Graphical User Interface of
the Application Under Test. GUI testing involves checking the screens with
the controls like menus, buttons, icons, and all types of bars - toolbar, menu
bar, dialog boxes and windows, etc.
 GUI is what user sees. A user does not see the source code. The interface is
visible to the user. Especially the focus is on the design structure, images
that they are working properly or not.
Following are the GUI elements which can be used for interaction between the
user and application:

In addition to functionality, GUI testing evaluates design elements such

as layout, colors, fonts, font sizes, labels, text boxes, text formatting,
captions, buttons, lists, icons, links and content. GUI testing processes
can be either manual or automatic, and are often performed by third -
party companies, rather than developers or end users.
Example GUI Testing Test Cases
GUI Testing basically involves
Testing the size, position, width, height of the elements.
Testing of the error messages that are getting displayed.
Testing the different sections of the screen.
Testing of the font whether it is readable or not.
Testing of the screen in different resolutions with the help of zooming in and
zooming out like 640 x 480, 600x800, etc.
Testing the alignment of the texts and other elements like icons, buttons, etc.
are in proper place or not.
Testing the colors of the fonts.
Testing the colors of the error messages, warning messages.
Testing whether the image has good clarity or not.
Testing the alignment of the images
Testing of the spelling.
The user must not get frustrated while using the system interface.
Testing whether the interface is attractive or not.
Testing of the scrollbars according to the size of the page if any.
Testing of the disabled fields if any.
Testing of the size of the images.
Testing of the headings whether it is properly aligned or not.
Testing of the color of the hyperlink.
Testing Web Applications
 WEB TESTING, or website testing is checking your web application or
website for potential bugs before its made live and is accessible to general
public. Web Testing checks for functionality, usability, security, compatibility,
performance of the web application or website.
 During this stage issues such as that of web application security, the
functioning of the site, its access to handicapped as well as regular users
and its ability to handle traffic is checked.

How to test Web Application

In Software Engineering, the following testing types/technique may be performed depending on
your web testing requirements.

1. Functionality Testing of a Website: White Box Testing

Functionality Testing of a Website is a process that includes several testing parameters like user
interface, APIs, database testing, security testing, client and server testing and basic website
functionalities. Functional testing is very convenient and it allows users to perform both manual
and automated testing. It is performed to test the functionalities of each feature on the website.
Testing Web Applications

 Web based Testing Activities includes:

Test all links in your webpages are working correctly and make sure there are
no broken links. Links to be checked will include -
 Outgoing links
 Internal links
 Anchor Links
 MailTo Links
Test Forms are working as expected. This will include-
 Scripting checks on the form are working as expected. For example- if a user
does not fill a mandatory field in a form an error message is shown.
 Check default values are being populated
 Once submitted, the data in the forms is submitted to a live database or is
linked to a working email address
 Forms are optimally formatted for better readability
Testing Web Applications
Test Cookies are working as expected. Cookies are small files used by websites to
primarily remember active user sessions so you do not need to log in every time
you visit a website. Cookie Testing will include
 Testing cookies (sessions) are deleted either when cache is cleared or when they
reach their expiry.
 Delete cookies (sessions) and test that login credentials are asked for when you
next visit the site.
Test HTML and CSS to ensure that search engines can crawl your site easily. This
will include: Gray Box Testing
 Checking for Syntax Errors
 Readable Color Schemas
 Standard Compliance. Ensure standards such W3C, OASIS, IETF, ISO, ECMA,
or WS-I are followed.
Test business workflow- This will include
 Testing your end - to - end workflow/ business scenarios which takes the user
through a series of webpages to complete.
 Test negative scenarios as well, such that when a user executes an unexpected
step, appropriate error message or help is shown in your web application.
Testing Web Applications

2. Usability testing:
 Usability Testing has now become a vital part of any web based project. It can
be carried out by testers like you or a small focus group similar to the
target audience of the web application.
 Test the site Navigation:
 Menus, buttons or Links to different pages on your site should be easily visible
and consistent on all webpages
 Gratuitous use of bleeding edge technology
 Scrolling text, Marquees and constantly running animation
 Long Scrolling Pages
 Non standard link colors
 Outdated information
 Overly long download time
 Lack of navigation support
 Orphan pages
Testing Web Applications

3.Interface Testing:
 Three areas to be tested here are - Application, Web and Database
Server
 Application: Test requests are sent correctly to the Database and
output at the client side is displayed correctly. Errors if any must be
caught by the application and must be only shown to the
administrator and not the end user.
 Web Server: Test Web server is handling all application requests
without any service denial.
 Database Server: Make sure queries sent to the database give
expected results.
 Test system response when connection between the three
layers (Application, Web and Database) cannot be
established and appropriate message is shown to the end user.
Testing Web Applications

4. Database Testing:
 Database is one critical component of your web application and
stress must be laid to test it thoroughly. Testing activities will include-
 Test if any errors are shown while executing queries
 Data Integrity is maintained while creating, updating or deleting data
in database.
 Check response time of queries and fine tune them if necessary.
 Test data retrieved from your database is shown accurately in your
web application
Testing Web Applications
5. Compatibility testing.
 Compatibility tests ensures that your web application displays
correctly across different devices. This would include-
 Browser Compatibility Test: Same website in different browsers
will display differently. You need to test if your web application is
being displayed correctly across browsers, JavaScript, AJAX and
authentication is working fine. You may also check for Mobile
Browser Compatibility.
 The rendering of web elements like buttons, text fields etc. changes
with change in Operating System. Make sure your website works
fine for various combination of Operating systems such as Windows,
Linux, Mac and Browsers such as Firefox, Internet Explorer, Safari
etc.
Testing Web Applications
6. Performance Testing:
 This will ensure your site works under all loads. Software Testing
activities will include but not limited to -
 Website application response times at different connection speeds
 Load test your web application to determine its behavior under
normal and peak loads
 Stress test your web site to determine its break point when pushed
to beyond normal loads at peak time.
 Test if a crash occurs due to peak load, how does the site recover
from such an event
 Make sure optimization techniques like gzip compression, browser
and server side cache enabled to reduce load times
Testing Web Applications

7. Security testing:
 Security Testing is vital for e-commerce website that store sensitive
customer information like credit cards. Testing Activities will include-
 Test unauthorized access to secure pages should not be permitted
 Restricted files should not be downloadable without appropriate
access
 Check sessions are automatically killed after prolonged user
inactivity
 On use of SSL certificates, website should re-direct to encrypted SSL
pages.
Testing Web Applications
8. Crowd Testing:
 You will select a large number of people (crowd) to execute tests
which otherwise would have been executed a select group of people
in the company. Crowdsourced testing is an interesting and
upcoming concept and helps unravel many a unnoticed defects.

 As a Web-tester its important to note that web testing is quite an arduous

process and you are bound to come across many obstacles. One of the major
problems you will face is of course deadline pressure. Everything is always
needed yesterday! The number of times the code will need changing is
also taxing. Make sure you plan your work and know clearly what is
expected of you. Its best define all the tasks involved in your web testing
and then create a work chart for accurate estimates and planning.
Object Oriented Application Testing
 The process of testing object-oriented systems begins with a review of
the object-oriented analysis and design models. Once the code is written
object-oriented testing (OOT) begins by testing "in the small" with class
testing (class operations and collaborations). As classes are integrated
to become subsystems class collaboration problems are investigated.
Finally, use-cases from the OOA model are used to uncover software
validation errors.
 Test cases are developed to exercise the classes, their collaborations,
and behavior.
 OOT tends to focus more on integration problems than on unit testing.
 Unit testing in the OO context
o Smallest testable unit is the encapsulated class or object
o Similar to system testing of conventional software
o Do not test operations in isolation from one another
o Driven by class operations and state behavior, not algorithmic detail and
data flow across module interface
o Complete test coverage of a class involves
o Testing all operations associated with an object
o Setting and interrogating all object attributes
o Exercising the object in all possible states
State-based testing
 State-based testing is used to verify whether the methods (a procedure that is
executed by an object) of a class are interacting properly with each other. This
testing seeks to exercise the transitions among the states of objects based
upon the identified inputs.
 For this testing, finite-state machine (FSM) or state-transition diagram
representing the possible states of the object and how state transition occurs is
built. In addition, state-based testing generates test cases, which check
whether the method is able to change the state of object as expected. If any
method of the class does not change the object state as expected, the method
is said to contain errors.
 To perform state-based testing, a number of steps are followed, which are
listed below.
 Derive a new class from an existing class with some additional features, which
are used to examine and set the state of the object.
 Next, the test driver is written. This test driver contains a main program to
create an object, send messages to set the state of the object, send messages
to invoke methods of the class that is being tested and send messages to
check the final state of the object.
 Finally, stubs are written. These stubs call the untested methods.
Fault-based Testing

 Fault-based testing is used to determine or uncover a set of

plausible faults or the focus of tester in this testing is to detect
the presence of possible faults.
 Fault-based testing starts by examining the analysis and
design models of OO software as these models may provide an
idea of problems in the implementation of software.
 With the knowledge of system under test and experience in
the application domain, tester designs test cases where each
test case targets to uncover some particular faults.
 The effectiveness of this testing depends highly on tester
experience in application domain and the system under test.
 This is because if he fails to perceive real faults in the system
to be plausible, testing may leave many faults undetected.
 However, examining analysis and design models may enable
tester to detect large number of errors with less effort.
 As testing only proves the existence and not the absence of
errors, this testing approach is considered to be an effective
method and hence is often used when security or safety of a
system is to be tested.
 Integration testing applied for OO software targets to
uncover the possible faults in both operation calls and
various types of messages (like a message sent to invoke an
object).
 These faults may be unexpected outputs, incorrect messages
or operations, and incorrect invocation.
 The faults can be recognized by determining the behavior of
all operations performed to invoke the methods of a class.
Scenario-based Testing

 Scenario-based testing is used to detect errors that are caused due to

incorrect specifications and improper interactions among various
segments of the software.
 Incorrect interactions often lead to incorrect outputs that can cause
malfunctioning of some segments of the software.
 The use of scenarios in testing is a common way of describing how a
user might accomplish a task or achieve a goal within a specific context
or environment.
 Note that these scenarios are more context- and user specific instead of
being product-specific. Generally, the structure of a scenario includes
the following points.
 A condition under which the scenario runs.
 A goal to achieve, which can also be a name of the scenario.
 A set of steps of actions.
 An end condition at which the goal is achieved.
 A possible set of extensions written as scenario fragments.
 Scenario- based testing combines all the classes that support a use-
case (scenarios are subset of use-cases) and executes a test case to
test them. Execution of all the test cases ensures that all methods in all
the classes are executed at least once during testing.
 However, testing all the objects (present in the classes combined
together) collectively is difficult. Thus, rather than testing all objects
collectively, they are tested using either top-down or bottom-up
integration approach.
 This testing is considered to be the most effective method as scenarios
can be organized in such a manner that the most likely scenarios are
tested first with unusual or exceptional scenarios considered later in the
testing process.
 This satisfies a fundamental principle of testing that most testing effort
should be devoted to those paths of the system that are mostly used.
Types of UML Diagrams
The current UML standards call for 13 different types of diagrams: class, activity,
object, use case, sequence, package, state, component, communication,
composite structure, interaction overview, timing, and deployment.
These diagrams are organized into two distinct groups: structural diagrams and
behavioral or interaction diagrams.
Structural UML diagrams
Class diagram
Package diagram
Object diagram
Component diagram
Composite structure diagram
Deployment diagram
Behavioral UML diagrams
Activity diagram
Sequence diagram
Use case diagram
State diagram
Communication diagram
Interaction overview diagram
Timing diagram
Class Diagram
Class diagrams are the backbone of almost every object-
oriented method, including UML. They describe the static
structure of a system
Package Diagram
Package diagrams are a subset of class diagrams, but developers sometimes
treat them as a separate technique. Package diagrams organize elements of a
system into related groups to minimize dependencies between packages.
Object Diagram
Object diagrams describe the static structure of a system at a particular time. They
can be used to test class diagrams for accuracy.
Use Case Diagram
Use case diagrams model the functionality of a system using actors and use
cases
Activity Diagram
Activity diagrams illustrate the dynamic nature of a system by modeling the flow of
control from activity to activity. An activity represents an operation on some class in
the system that results in a change in the state of the system. Typically, activity
diagrams are used to model workflow or business processes and internal operation.
Sequence Diagram
Sequence diagrams describe interactions among classes in terms of an exchange of
messages over time
State Diagram
Statechart diagrams, now known as state machine diagrams and state diagrams
describe the dynamic behavior of a system in response to external stimuli. State
diagrams are especially useful in modeling reactive objects whose states are
triggered by specific events
Component Diagram
Component diagrams describe the organization of physical software components,
including source code, run-time (binary) code, and executables.
Interaction Overview Diagram

Interaction overview diagrams are a combination of activity and sequence

diagrams. They model a sequence of actions and let you deconstruct more
complex interactions into manageable occurrences. You should use the same
notation on interaction overview diagrams that you would see on an activity
diagram.

Timing Diagram

A timing diagram is a type of behavioral or interaction UML diagram that

focuses on processes that take place during a specific period of time. They're
a special instance of a sequence diagram, except time is shown to increase
from left to right instead of top down.

Communication Diagram

Communication diagrams model the interactions between objects in

sequence. They describe both the static structure and the dynamic behavior
of a system. In many ways, a communication diagram is a simplified version
of a collaboration diagram introduced in UML 2.0.