Unit 4 – Domain Testing

Domain Testing U3

We will see in this part of Unit 3:

• Domains & Paths

• Nice & Ugly Domains

• Domain Testing

• Domains & Interfaces Testing

• Domains & Testability

ref boris beizer 2

Transaction-Flow Testing U3

Contents of the unit

• Domains
• Model, as a set, paths
• Predicates
• closure, dimensionality, assumptions
• Nice & ugly domains
• specified & implemented domains
• Nice domains – properties, testing implications
• Ugly domains, simplification by programmers & testers
• Domain testing
• Bugs & testing for those bugs
• Testing procedure
• Variations, tools, effectiveness
• Domains & Interface Testing
• Range, closure, span
• Interface range, domain compatibility
• Finding the values
• Domains & Testability
• Linearizing, coordinate transformations.
ref borisCanonical
beizer program form 3
Domain Testing - Domains & Paths U3

Domain Testing Model

D1
D2

INPUT CLASSIFY DO CASE 1 OUTPUT

D3

(x,y,z,…) Dn { Outcome }
DO CASE 2

ref boris beizer

(1, 2, 3, 4, 5,…)

DO CASE 3

DO CASE n

4
Domain Testing U3

Domain Testing

Views Programs as input data classifiers

ref boris beizer

Verifies if the classification is correct

5
Domain Testing - Domains & Paths U3

Domain Testing Model

Two Views

• Based on Specs

ref boris beizer

• Functional Testing

• Based on Implementation information

• Structural Technique
6
Domain Testing - Domains & Paths U3

Domain Testing Model

• Variables
• Simple combinations of two variables
• Numbers from - to +

ref boris beizer

• Input variables as numbers

• Loop-free Programs
7
Domain Testing - Domains & Paths U3

Domain Testing Model

• Structural Knowledge is not needed. Only Specs.

ref boris beizer

• For each Input Case,

• A Hypothetical path for Functional testing

• An Actual path for Structural testing

8
Domain Testing - Domains & Paths U3

Domain (simplified)

• A Single Connected Set of numbers D1

• No arbitrary discrete sets

• Defined by the boundaries

ref boris beizer

• One or more boundaries
• Specified by predicates
• Bugs likely at the boundaries
• One or more variables

9
Domain Testing - Domains & Paths U3

Predicates

• Interpretation

• Structural Testing - CFG

ref boris beizer

• Functional Testing - DFG

• Specifies the Domain Boundary

• n Predicates in Sequence  2n domains

• Or, just two domains A .AND. B .AND. C 10
Domain Testing - Domains & Paths U3

Paths

• At least One PATH thru the Program

• More paths for disconnected domains

ref boris beizer

11
Domain Testing - Domains & Paths U3

Summary:

• Domain for a loop-free program corresponds to a set of

numbers defined over the input vector

ref boris beizer

• For every domain, there is at least one path thru the routine,
along which that domain’s processing is done

• The set of interpreted predicates traversed on that path (ie.,

the path’s predicate expression) defines the domain’s
boundaries. 12
Domain Testing - Domains & Paths - Domain Closure U3

Boundary: Closed / Open

X >= MIN X <= MAX

MAX
MIN
D1 D2 D3

Closed

ref boris beizer

X >= MIN D2 X < MAX D3
D1

Closed Open
MIN MAX
13
Domain Testing - Domains & Paths - Domain Closure
Boundary: Closed / Open

MAX
MIN
D2 D3
D1

ref boris beizer

Open
X > MIN X < MAX

14
Domain Testing - Domains & Paths - Domain Dimensionality U3

• One dimension per variable

• At least one predicate

ref boris beizer

• Slices thru previously defined Domain

• Slicing Boundary

• N-spaces are cut by Hyperplanes 15

Domain Testing - Domains & Paths – Bug Assumptions U3

• Processing is OK. Domain definition may be wrong.

 Boundaries are wrong.

 Predicates are wrong.

ref boris beizer

• Once input vector is set on the right path, it’s correctly
processed.

• More bugs causing domain errors … 16

Domain Testing - Domains & Paths – Bug Assumptions.. U3

• Double-zero representation

• Floating point zero check

ref boris beizer

• Contradictory domains

defined
17
• Ambiguous domains
undefined
Domain Testing - Domains & Paths – Bug Assumptions.. U3

• Over-specified domains
X > 3 .AND. X < 2 .AND. Y > 3

• Boundary errors

ref boris beizer

Boundary closure

18
Shifted
Domain Testing - Domains & Paths - Bug Assumptions U3

• Boundary errors….

Tilted

ref boris beizer

Missing

19
Extra boundary
Domain Testing - Domains & Paths - Bug Assumptions U3

X >= 3

• Closure Reversal

ref boris beizer

• Faulty Logic

Simplification of compound predicates 20

Domain Testing - Domains & Paths - Bug Assumptions U3

ref boris beizer

21
Strategy for domain testing in 2-dim
Domain Testing - Domains & Paths - Restrictions U3

In General…

1. Coincidental Correctness

DT cannot detect

ref boris beizer

Example F1(x) : +1

D1

• Representative outcome

Partition testing D2

F2(x) : -1 22
Input Equivalence
Domain Testing - Domains & Paths U3

Domain Testing Model

Function f1
D1

INPUT CLASSIFY DO CASE 1 OUTPUT

D2
(x, y, z, …) { Outcome }
DO CASE 2

ref boris beizer

(1, 2, 3, 4, 5,…) f2

D3
DO CASE 3 f3

Dn
f4
DO CASE n

23
Domain Testing - Domains & Paths - - Restrictions U3

3. Simple boundaries & Compound predicates

D1

X X >=0 .AND. X <= 16

0 16

ref boris beizer

D1
Y Y = 1 .AND. X >=0 .AND. X <= 16
0 16

24
Domain Testing - Domains & Paths - - Restrictions U3

Compound predicates….

• Impact of .OR.

• Concave, Disconnected

ref boris beizer

• Adjacent domains with same function

• Example

ABC + DEF
25
• Eliminate compound predicates
Domain Testing - Domains & Paths - Restrictions U3

4. Functional Homogeneity of Bugs

• Functional form still Retained

ref boris beizer

a x + b y >= c

• Bugs are only in a, b, c

26
Domain Testing - Domains & Paths - - Restrictions U3

5.Linear Vector Space

• Linear boundary predicate, Interpreted

• Simple relational operators

ref boris beizer

• Conversion to linear vector space

• 2-d Polar co-ordinates

• Polynomials

• Problems with Non-linear Boundaries 27

Domain Testing - Domains & Paths - Restrictions U3

6. Loop-free Software

Predicate for each iteration

ref boris beizer

Loop over the entire transaction

Definite loop
28
Domain Testing - Domains & Paths – Nice & Ugly domains U3

Nice Domains
Requirements

• Bugs  ill-defined domains

ref boris beizer

Before DT

• Analyze specs
29
• Make the Boundary Specs Consistent & Complete
Domain Testing - Domains & Paths – Nice domains U3

Implemented Domains

• Complete, Consistent & Process all inputs

ref boris beizer

Specified Domains

• Incomplete, Inconsistent
30
Programmer’s / Designer’s Effort
Domain Testing - Domains & Paths – Nice domains U3

Nice Domains

• Linear, Complete, Systematic, Orthogonal, Consistently Closed,

& Convex

U1 U2

ref boris beizer

V1 D11 D12

V2 D21 D22

31
Domain Testing - Domains & Paths – Nice domains U3

Nice Domains

Advantages

ref boris beizer

• Ease of DT

• Fewer Bug Occurrences

32
Domain Testing - Domains & Paths – Nice domains U3

Nice Domains
Boundaries are

5. Closure consistency
1.Linear

ref boris beizer

1.Complete
6. Convex
2.Systematic

1.Orthogonal 7. Simply connected

33
Domain Testing - Domains & Paths – Nice domains U3

1.Linear Boundaries

Interpreted linear inequalities

n-dim Hyperplane:

ref boris beizer

n+1 Points

n+1 + 1 Test Cases

Non-Linear
34
• Transform
Domain Testing - Domains & Paths – Nice domains U3

2.Complete Boundaries

• Span the total number space (-, +)

• One set of Tests

ref boris beizer

Incomplete…

• Reasons
35
• Tests
Domain Testing - Domains & Paths - Nice domains U3

3. Systematic Boundaries

• Linear Inequalities differing by a constant

fj (X) ≥ kj or, fj (X) ≥ g (j, c) g (j, c) = j + k * c

ref boris beizer

• Parallel lines
• Identical Sectors in a Circle

• DT
• Test a domain  Tests for other Domains 36
Domain Testing - Domains & Paths - Nice domains U3

4. Orthogonal Boundaries

• Two boundaries or, boundary sets Uj

• Parallel to axes

ref boris beizer

Vj

• DT

• Each Set Independently

• # Tests  O (n) 37
Domain Testing - Domains & Paths - Nice domains U3

Orthogonal Boundaries

• Tilted sets
• transformation
• Test cases: O(n)

ref boris beizer

• Concentric circles with radial lines

• Rectangular coordinates
• Polar
r ≥ aj .AND. r < aj+1 .AND.
38
 ≥ j .AND.  < j+1
Domain Testing - Domains & Paths - Nice domains U3

Non-Orthogonal Boundaries

• Test Intersections  O ( n2 ) + O (n)

ref boris beizer

39
Domain Testing - Domains & Paths - Nice domains U3

5. Closure Consistency

• A Simple pattern in all boundary closures

• Example

ref boris beizer

• Same relational operator for systematic boundaries
D3 D1 D2 D4

40
Domain Testing - Domains & Paths - Nice domains U3

6. Convex Domain

• Line joining any two points lies with in the domain

• DT
• n on-points & 1 off-pt

ref boris beizer

Concave

• “ But, However, Except, Or … “ in Specs

• Handle with special care
41
Domain Testing - Domains & Paths - Nice domains U3

7. Simply Connected Domain

• In a single piece D2 D1 D3

D2
D1

ref boris beizer

• 2 complementary domains

• D1: Convex  D2: Concave, not-Connected

• Programmer / Designer
42
• Convex part First
Domain Testing - Domains & Paths – Ugly Domains U3

Generally,

From Bad Specs

ref boris beizer

Programmer / Designer Simplifies => Ugly to Good

possibility of Introduction of more bugs ?!?

43
Domain Testing - Domains & Paths – Ugly Domains U3

Causes

1. Non-linear Boundaries

ref boris beizer

2. Ambiguities & Contradictions

3. Simplifying the topology

44
4. Rectifying boundary closures
Domain Testing - Domains & Paths – Ugly Domains U3

1. Non-linear Boundaries

• Transform

ref boris beizer

45
Domain Testing - Domains & Paths – Ugly Domains U3

2. Ambiguities

• Holes in input vector space.

• Missing boundary

ref boris beizer

• Detected by Specification languages & tools.

46
Domain Testing - Domains & Paths – Ugly Domains U3

2. Contradictions..

• Overlapping of

• Domain Specs

ref boris beizer

• Closure Specs

D3

D1 D2

47
Domain Testing - Domains & Paths – Ugly Domains U3

3. Simplifying the Topology….

• Complexity !

ref boris beizer

• Concavity, Holes, Disconnectedness

48
Domain Testing - Domains & Paths – Ugly Domains U3

3. Simplifying the Topology….

• Smoothing out concavity

ref boris beizer

• Filling in Holes

49
Domain Testing - Domains & Paths – Ugly Domains U3

3. Simplifying the Topology….

• Joining the pieces

ref boris beizer

Correct:

• Connect disconnected boundary segments

• Extend boundaries to infinity 50

Domain Testing - Domains & Paths – Ugly Domains U3

4. Rectifying Boundary Closures.

• make closures in one direction for parallel boundaries with

closures in both directions

• Force a Bounding Hyperplane to belong to the Domain.

ref boris beizer

51
Consistent Direction Inclusion / Exclusion Consistency
Domain Testing - Domains & Paths – Domain Testing U3

General DT Strategy

1. Select test points near the boundaries.

2. Define test strategy for each possible bug related to boundary

ref boris beizer

3. Test points for a domain useful to test its adjacent domain.

4. Run the tests. By post test analysis determine if any boundaries

are faulty & if so how?

5. Run enough tests to verify every boundary of every domain 52

Domain Testing - Domains & Paths – Domain Testing U3

DT for Specific Domain Bugs

Generally,

• Interior point

ref boris beizer

• Exterior point

• Epsilon neighborhood

• Extreme point

• On point 53
Domain Testing - Domains & Paths – Domain Testing U3

DT for Specific Domain Bugs

Off Points

Extreme
point

ref boris beizer

Boundary point

Epsilon neighborhood
On Points

54
Domain D1
Domain Testing - Domains & Paths – Domain Testing U3

1. 1-d Domains

2. 2-d Domains

ref boris beizer

3. Equality & inequality Predicates

4. Random Testing

55
5. Testing n-dimensional Domains
Domain Testing - Domains & Paths – Domain Testing U3

Testing 1-d Domains : Bugs with open boundaries

B A B A
Closure Bug
x x

ref boris beizer

B A
Shift left Bug
x1 x

B A
Shift Right Bug
x x1
56
Domain Testing - Domains & Paths – Domain Testing U3

Testing 1-d Domains : Bugs with open boundaries

B A B A
Missing Boundary x
x x

ref boris beizer

Extra Boundary B A C
x x
x x1

Bugs with Closed Boundaries : Similar to the above

57
Domain Testing - Domains & Paths – Domain Testing U3

2. Testing 2-d Domains

• Closure bug

• Boundary Shift : up / down

ref boris beizer

• Tilted Boundary

• Extra Boundary

• Missing Boundary
58
Domain Testing - Domains & Paths – Domain Testing U3

ref boris beizer

2 n : Domains share tests
3 n : no sharing of tests by domains
59
Strategy for domain testing in 2-dim
Domain Testing - Domains & Paths – Domain Testing U3

3. Equality & Inequality Predicates

• An Equality predicate defines a line in 2-d

a
A

ref boris beizer

c c’
c
d
B
b

To avoid bugs
60
Domain Testing - Domains & Paths – Domain Testing U3

4. Random Testing

• A Point in the center : verifies computation

ref boris beizer

61
Domain Testing - Domains & Paths – Domain Testing U3

5. Testing n-Dimensional Domains (strategy)

n-dimensions, p boundary segments

• (n+1)*p test cases : n on points & 1 off point

ref boris beizer

• Extreme pt shared : 2 * p points

• Equalities over m-dimensions create a subspace of n-m

dimensions
• Orthogonal domains with consistent boundary closures,
orthogonal to the axes & complete boundaries
• Independent testing 62
Domain Testing - Domains & Paths – Domain Testing U3

Procedure for solving for values

Simple procedure. Need tools.

1. Identify input variables

ref boris beizer

2. Identify variables which appear in domain-defining predicates,
such as control-flow predicates

63
Domain Testing - Domains & Paths – Domain Testing U3

Procedure

3. Interpret all domain predicates in terms of input variables:

• Transform non-linear to linear
• Find data flow path

ref boris beizer

4. Predicate expression with p # predicates.
Find # domains : < 2 p

5. Solve inequalities for extreme points

6. Use extreme points to solve for nearby on points …

64
Domain Testing - Domains & Paths – Domain Testing U3

Effectiveness

• Cost effective

• Bugs on boundaries, extreme points

ref boris beizer

• Hardware logic testing – tool intensive

65
Domain Testing - Domains & Paths – Domain Testing U3

Domains & Interface Testing

• Domain

• Range

ref boris beizer

Range
Variable Domain
Function /
Classify
Routine

66
Domain Testing - Domains & Paths – Domain Testing U3

Domains & Interface Testing

Variable
Range for Domain for Range for
Routine 1 Routine1
Routine 2 Routine2
Domain Routine2

ref boris beizer

• Span compatibility

67
Domain Testing - Domains & Paths – Domain Testing U3

Domains & Interface Testing

• Closure compatibility

ref boris beizer

68
Domain Testing - Domains & Paths – Domain Testing U3

Interface Range/Domain Compatibility Testing

• Test each var independently

ref boris beizer

• Find an inverse function

• Build a super domain

69
Domain Testing - Domains & Paths – Domain Testing U3

Finding the values / solving inequalities

ref boris beizer

70
Domain Testing - Domains & Paths – Domain Testing U3

Domains and Testability

• Linearizing transformations

• Polynomial
• Rectangular to polar

ref boris beizer

• Generic & rational => Taylor series

71
Domain Testing - Domains & Paths – Domain Testing U3

Domains and Testability

Perform transformations for better testing.

• Co-ordinate transformations

ref boris beizer

72
Domain Testing - Domains & Paths – Domain Testing U3

Domains and Testability

• Canonical Program Form

ref boris beizer

73
Domain Testing – Questions from the previous year’s exams U3

1. Explain Nice & Ugly Domains.

2. What is domain testing? Discuss applications of domain testing

3. Explain the domain boundary bugs for two dimensional domains

ref boris beizer

4. What is the purpose of Domain Testing? Give its schematic representation

5. Define the following concepts.

1. Domains 2. Domain closure 3. Domain dimensionality 4.
Bug assumptions for Domain Testing

6. Explain simple domain boundaries & compound predicates.

74
7. Classify what can go wrong with boundaries, then define a test strategy for
each case in domain testing.
• 1. Discuss with example the equal - span range/Doman compatibility bugs.(8 M)*
• 2. Discuss in detail about testability of Domains.(8 M)*
• 3. What is meant by Domain Dimensionality. (8 M)
• 4. What is meant by nice - domain? Give an example for nice two – dimensional domain.
• (8 M).*
• 5. Discuss (8 M)
• i. Linear domain boundaries ii. Non linear domain boundaries
• iii. Complete domain boundaries iv. Incomplete domain boundaries

ref boris beizer

• 6. Explain various properties related to Ugly-domains. (8 M) *
• 7. State and Explain various restrictions at domain testing processes. (10 M)
• 8. What is meant by domain testing? Discuss the various applications of domain testing?
• (10 M) *
• 9. With a neat diagram, explain the schematic representation of domain testing.(6 M)
• 10.Explain how one-dimensional domains are tested? (10 M) 75
• 11.Discuss in detail the domains and interface testing. (16 M)
Domain Testing U-3

To Unit 5…Paths, Path Products & Regular Expressions

ref boris beizer

76