FUNCTIONAL VERIFICATION

Anantharaj TV
Staff ASIC Verification Engineer
SanDisk India
Bangalore

1 09/07/2013 Functional Verification

Disclaimer

“ALL THE LOGOS,INFORMATIONS ON PRODUCTS &

PROCESSES ARE COPYRIGHTED & PROTECTED
UNDER THE RESPECTIVE COMPANY’S LAND LAW”

“INTENDED USAGE IS STRICTLY FOR EDUCATIONAL

PURPOSE AND NOT FOR ANY COMMERCIAL
PURPOSES”
“PRESENTATION MATERIAL SHOULD NOT BE COPIED
OR DISTRIBUTED”

2 09/07/2013 Functional Verification

First Monolithic IC (1958)

JACK KILBY

3 09/07/2013 Functional Verification

Agenda

Introduction to Logic Verification

Where does it fits?
What is Verification?
Logic Verification is ASIC Development Cycle
Technology Cycle
Verification Complexity
Verification Gap
Modeling Verification Process
Types of Verification
Is Testing a Verification?
Verification Challenges

4 09/07/2013 Functional Verification

Introduction to Logic Verification

5 09/07/2013 Functional Verification

Where Does it Fits?

?
6 09/07/2013 Functional Verification
Architecture Design

Logic Design &

Verification

Logic Synthesis &

Verification

Physical Design &

Verification

Fabrication & Testing

7 09/07/2013 Functional Verification

Levels of Abstraction

8 09/07/2013 Functional Verification

What is Verification?

9 09/07/2013 Functional Verification

What is Verification?

Verification is a process used to

demonstrate the functional correctness of
a design.
Verification is a stage to confirm the
expected functional behavior of a design.
Verification is a comparison between
actual and expected behaviors.

10 09/07/2013 Functional Verification

Verification in our daily life!!

Checking whether the door is locked or

not?
Checking whether the mobile phone is in
your pocket or not?
Checking on your wallet?
Thinking whether I knew anything about
Logic verification?

11 09/07/2013 Functional Verification

Agenda PART I

Introduction to Logic Verification

Where does it fits?
What is Verification?
Logic Verification is ASIC Development
Cycle

12 09/07/2013 Functional Verification

Logic Verification in ASIC
Development Cycle

13 09/07/2013 Functional Verification

A major bottleneck in ASIC development

cycle, consuming up-to 70% of the time &
resources
Logic verification is a function of both design
size and test vector size.
As the design size grows, the required test
vector size needed to functionally verify it
also grows.
14 09/07/2013 Functional Verification
A general rule of thumb:
– every 10x increase in design size 10x increase
in simulation vectors
– 100x increase in
verification complexity
Recently all major EDA companies in the electronic
sector are aggressively targeting the verification
process with new and better Hardware Verification
Languages (HVLs), EDA tools and Methodologies.

15 09/07/2013 Functional Verification

Below figure shows the statistics of the SOC designs in terms

of design complexity (logic gates), design time (engineer
years), and verification complexity (simulation vectors).

16 09/07/2013 Functional Verification

17 09/07/2013 Functional Verification

18 09/07/2013 Functional Verification

19 09/07/2013 Functional Verification

What do we verify?

"verification” is pre-silicon testing (Verilog/VHDL

simulations)
“validation” is post-silicon testing (testing silicon on
boards in the laboratory)
The chip specification is interpreted correctly
(typically through documentation and sometimes
modeling).
The interpretation is captured and implemented
correctly (typically using HDL) and synthesized into
silicon and packaged as a chip.

20 09/07/2013 Functional Verification

Agenda PART I

Introduction to Logic Verification

Where does it fits?
What is Verification?
Logic Verification is ASIC Development
Cycle
Technology Cycle

21 09/07/2013 Functional Verification

Technology Cycle

22 09/07/2013 Functional Verification

Technology Cycle

23 09/07/2013 Functional Verification

Technology Cycle

Following an exponential increase in the logic

functionality in designs, a linear increase in number
of design engineers was not adequate to reduce
design time.
To solve this problem, the electronic design
automation (EDA) industry stepped in to introduce
the concept of design abstraction through
automation.
Language-based solutions such as Verilog and
VHDL were introduced. (IEEE Standards, 1076-2002
and 1364-2001)

24 09/07/2013 Functional Verification

Technology Cycle

The latest languages being widely accepted

and supported by EDA world are SystemC
and SystemVerilog. (IEEE standards, 1666-
2005 and 1800-2005)
Design bottleneck has been overcome to
some extent thanks to the productivity gains
through the use of EDA tools.
Now the engineers focus is on solving the
effects of the verification bottleneck.
25 09/07/2013 Functional Verification

Agenda PART I

Introduction to Logic Verification

Where does it fits?
What is Verification?
Logic Verification is ASIC Development
Cycle
Technology Cycle
Verification Complexity

26 09/07/2013 Functional Verification

Verification Complexity

27 09/07/2013 Functional Verification

Verification Complexity

Increase in functional complexity because of the

heterogeneous nature of designs today; for example,
co-existence of hardware and software, analog and
digital.
To increase verification productivity, the EDA industry
came up with a solution similar to what was used to
solve the design bottleneck - the concept of
abstraction.
High-level language constructs were embedded into
Verilog and VHDL to help in verification; these
included constructs such as tasks, threading (fork,
join) and control structures.

28 09/07/2013 Functional Verification

Verification Complexity

However, these constructs were not

synthesizable and hence not used by
designers as part of actual design code.
As complexity continued to grow, new
verification languages & technologies were
created and introduced that could verify
complex designs at various levels of
abstraction.

29 09/07/2013 Functional Verification

Verification Gap

30 09/07/2013 Functional Verification

Verification Gap

31 09/07/2013 Functional Verification

Reducing Verification Time!

Effective Implementation
Visible Functionality
Parallelism
Higher Level of Abstraction
Automation

32 09/07/2013 Functional Verification

Agenda PART I

Introduction to Logic Verification

Where does it fits?
What is Verification?
Logic Verification is ASIC Development Cycle
Technology Cycle
Verification Complexity
Verification Gap
Modeling Verification Process
Types of Verification

33 09/07/2013 Functional Verification

Types of Verification

34 09/07/2013 Functional Verification

Types of Verification

Formal Verification
Logic or Functional Verification

35 09/07/2013 Functional Verification

Formal Verification

Often it is misunderstood that NO need to

develop test benches or test vectors
Divided into
– Equivalence checking &
– Model Checking

36 09/07/2013 Functional Verification

Equivalence Checking

•Mathematically proves both the input & output are logically equivalent,
that the transformation preserved its functionality
•Compares Boolean & Sequential logic functions, without mapping any
technology specifications.

37 09/07/2013 Functional Verification

Model Checking

Assertions or characteristics of a design are proven or dis-

proven
Model all the functional behaviors of the design & prove it.

38 09/07/2013 Functional Verification

Logic Verification

Design is implemented using logic gates

This logic implementation is proved against
the design specification
Divided into
– Black Box Verification
– White Box Verification
– Grey Box Verification

39 09/07/2013 Functional Verification

Black Box Verification

Zero visibility & controllability of the design

Needs to verify the design without any
knowledge on the actual implementation
Needs to develop test bench & test vectors
using the limited available interfaces &
variables

40 09/07/2013 Functional Verification

White Box Verification

Full visibility & controllability of the design

Efficiently verify the design with the
knowledge on the actual implementation &
internal structures
Easy to develop test bench & test vectors

41 09/07/2013 Functional Verification

Grey Box Verification

Compromise between Black & White Box

verification approaches
Limited visibility & controllability of the design
Limited knowledge on the actual
implementation & internal structures

42 09/07/2013 Functional Verification

Agenda PART I

Introduction to Logic Verification

Where does it fits?
What is Verification?
Logic Verification is ASIC Development Cycle
Technology Cycle
Verification Complexity
Verification Gap
Modeling Verification Process
Types of Verification
Is Testing a Verification?

43 09/07/2013 Functional Verification

Is Testing a Verification?

44 09/07/2013 Functional Verification

Is Testing a Verification?

Testing ensures that a design is

manufactured correctly
Verification ensures that logical
implementation of the design meet its
functional intent

45 09/07/2013 Functional Verification

Is Testing a Verification?

46 09/07/2013 Functional Verification

Verification Challenges

47 09/07/2013 Functional Verification

Verification Challenges

The verification engineer faces four major

challenges:
– dealing with enormous state space size,
– detecting incorrect behavior,
– lack of a golden reference model and
– lack of a comprehensive functional coverage
metric.

48 09/07/2013 Functional Verification

Above figure summarizes a study of the pre-silicon
logic bugs found in the Intel IA32 family of micro-
architectures.
This trend again shows an exponential increase in
the number of logic bugs: a growth rate of 300-
400% from one generation to the next.
The bug rate is linearly proportional to the number of
lines of structural RTL code in each design,
indicating a roughly constant density

49 09/07/2013 Functional Verification

where do all these bugs come

from?

50 09/07/2013 Functional Verification

51 09/07/2013 Functional Verification

Bugs Rate

52 09/07/2013 Functional Verification

Agenda PART I

Introduction to Logic Verification

Where does it fits?
What is Verification?
Logic Verification is ASIC Development Cycle
Technology Cycle
Verification Complexity
Verification Gap
Modeling Verification Process
Types of Verification
Is Testing a Verification?
Verification Challenges

53 09/07/2013 Functional Verification

Any Q’s?

54 09/07/2013 Functional Verification

End of Part I

55 09/07/2013 Functional Verification

Verification Tools

56 09/07/2013 Functional Verification

Linting

– Named after a UNIX utility that parses C program

– Identifies common mistakes a programmer make
– Identifies real problems like mismatched data
types between arguments of a function call or
incorrect bus width assignments
– Performed long before LOGIC VERIFICATION

57 09/07/2013 Functional Verification

Code Reviews

– Identifies functional & coding style errors

– Source code developed by a programmer is
reviewed by one or more peers
– Similar to code reviews in software industry
– Performed long before LOGIC VERIFICATION

58 09/07/2013 Functional Verification

Simulators

59 09/07/2013 Functional Verification

Simulators

Most common & known verification tools

Simulates the logical functionality of the
design in an artificial world
Simulators requires stimuli to the design, so
that they can generate responses from the
design

60 09/07/2013 Functional Verification

Simulation-based Verification

Execute the system in parallel with a reference

model…

01110101010100000000011101011011011110111

…with respect to some input sequences.

test as exhaustive
suite as possible
61 09/07/2013 Functional Verification

Types of Simulators

Event Driven
– Only executes a model when any of the input
changes (retains old state of response)
– Improved simulator efficiency

62 09/07/2013 Functional Verification

Event Driven

XOR gate viewed as a black box by the event driven simulator

63 09/07/2013 Functional Verification

Cycle Based Simulators

Only executes a model at a predefined cycle

time (like every positive edge of a clock
cycle)
Ignores all the intermediate changes, even if
any of the input changes

64 09/07/2013 Functional Verification

Cycle Based

65 09/07/2013 Functional Verification

Cycle Based

That sequential circuit is viewed as shown

below, by the cycle based simulator:

66 09/07/2013 Functional Verification

Co-Simulators

Includes both the Even Driven & Cycle

Based features
Very few designs are 100% synchronous in
real world
Synchronous portion of the design simulated
using Cycle Based simulator, while the rest
of the design are simulated with Event Driven
simulator
67 09/07/2013 Functional Verification

Co-Simulators

68 09/07/2013 Functional Verification

Code Coverage

69 09/07/2013 Functional Verification

Code Coverage

A Methodology widely used in Software

Industry
Helps to achieve 100% verification
completeness goals
Provides metrics to answer whether my
design is functionally correct even after all
my test cases/test benches are simulated
successfully!
70 09/07/2013 Functional Verification
Code Coverage Process

71 09/07/2013 Functional Verification

Code Coverage (contd.)

Identifies whether all the codes in the design

are excited by the stimuli from the verification
process
Any portion of the code that is unexcited may
contain bugs!
Types are Block, Path & Expression
coverages

72 09/07/2013 Functional Verification

Block Coverage

Measures how much lines in the code are

executed?

73 09/07/2013 Functional Verification

Path Coverage

Measures how many possible ways of

executing a sequence of code?

74 09/07/2013 Functional Verification

Expression Coverage

Measures how many various ways, the paths

in the code are executed?

75 09/07/2013 Functional Verification

Functional Coverage

76 09/07/2013 Functional Verification

Functional Coverage

Measures how many functionalities in the design are

executed?
Functional Coverage measures the design intent
Code Coverage measures the design
implementation
For example, the Functional Coverage catches a
Bug on missing a portion of RTL code where as
Code Coverage missed it (B’coz it was NOT at all
implemented)

77 09/07/2013 Functional Verification

Coverage Feedback

Provides feedback to the test bench on the

functionalities covered & left uncovered with
all the stimuli applied so far

78 09/07/2013 Functional Verification

Coverage Convergence

79 09/07/2013 Functional Verification

Coverage Comparison

80 09/07/2013 Functional Verification

Coverage in simulation-based verification

-Measure the exhaustiveness of the test suite.

-Examples: number of code lines visited.
number of latches that change value.

Plays an important role in the

design validation efforts.

Monitor progress.
More tests?

81 Direct simulation09/07/2013
towards unexplored areas.
Functional Verification

Agenda PART II

Verification Measurability
Verification as a Hypothetical Testing!
Verification Tools
– Linting
– Code Reviews
– Simulators
– Code Coverage
– Functional Coverage
– HVLs
82 09/07/2013 Functional Verification
Hardware Verification Languages
(HVLs)

83 09/07/2013 Functional Verification

Hardware Verification Languages

(HVLs)

Verilog HDL & VHDL are hardware

description/design languages (HDLs) NOT
verification languages
Need to increase the level of abstraction in
the verification process
Similar to using C language instead of
assembly codes for Micro-controllers to
increase the productivity
84 09/07/2013 Functional Verification
Examples of HVLs

Languages:
– System Verilog from Accellara IEEE
– e/Specman from Verisity
– Vera from Synopsys
Assertions:
– Property Specification Language (PSL)
– System Verilog Assertions (SVA)

85 09/07/2013 Functional Verification

Revision Control

Controlling the version of the code that is

being verified, developed & synthesized
Need to make sure that the recently
developed RTL code is verified 100% & the
same is taped-out

86 09/07/2013 Functional Verification

Issue Tracking

Verification engineer founds a Bug & report the issue to the

design team
Needs to track whether that issue is resolved in the design &
that RTL modification doesn’t cause any more issues

87 09/07/2013 Functional Verification

Bus Functional Models (BFMs)

Develop reference model mimicking the functional

behavior of the design at a higher level of abstraction
Stimuli are applied to both the Design Under
Verification (DUV) & the Reference Model
Responses from DUV & Model are constantly
monitored & compared for any discrepancies
These reference models are also known as BFMs

88 09/07/2013 Functional Verification

BFMs

89 09/07/2013 Functional Verification

Agenda PART II

Verification Measurability
Verification as a Hypothetical Testing!
Verification Tools
– Linting
– Code Reviews
– Simulators
– Code Coverage
– Functional Coverage
– HVLs
– Revision Control
– Issue tracking
– BFMs
Levels of Verification
90 09/07/2013 Functional Verification
Levels of Verification

91 09/07/2013 Functional Verification

Verification Strategy

92 09/07/2013 Functional Verification

Verification Strategy

Need to define a strategy for performing

verification on a design functionality
Depends on
– Levels of verification
– Types of test cases/ test benches

93 09/07/2013 Functional Verification

Directed Verification

Develop test cases for basic operations &

error/interrupt conditions
Stimuli are pre-determined & the responses
are expected

94 09/07/2013 Functional Verification

Directed Verification Progress

Linear & Steady progress on the verification

metrics

95 09/07/2013 Functional Verification

Directed Verification Coverage

96 09/07/2013 Functional Verification

Random Verification

Doesn’t mean to randomly apply ‘0’ or ‘1’ to each &

every input signal of the design
Sequence of valid operations are generated &
transferred randomly to the design
Creates a random valid input scenario, which is
beyond the imaginable solution space
Needs to provide input valid conditions for
generation of random stimuli -- Constraints
Known as Constrained Random Verification (CRV)

97 09/07/2013 Functional Verification

Random Verification Progress

Non-Linear & Speedy progress on the

verification metrics

98 09/07/2013 Functional Verification

Random Verification Coverage

99 09/07/2013 Functional Verification

Test Bench & its Approaches

10
0 09/07/2013 Functional Verification
Test Bench

A testbench is HDL code to verify a module

developed in HDL code
– Apply input vectors to module inputs
– Check module outputs
– Report errors to user
Advantage
– Portability - testbench will work on any HDL simulator
– Automatic checking - don't have to interpret waveform
– Expressability - can use the full semantics of HDL to:
generate input vectors (possibly from input file)
check output vectors
control simulation
10
1 09/07/2013 Functional Verification

Test Bench Environment

Test bench Module

Module Instance:
Device
Under
Verification
(DUV)

10
2 09/07/2013 Functional Verification
Test Bench Approaches

10
3 09/07/2013 Functional Verification

Visual Inspection

Test bench

Device
Waveform Viewer
Stimulus under
OR
Generator Verification
Text Output
(DUV)

10
4 09/07/2013 Functional Verification
Output Comparison

Test bench
“Gold”
Vectors
Reference
Model

Output Error/Status
Comparator Messages
Device
Stimulus under
Generator Verification
(DUV)

10
5 09/07/2013 Functional Verification

Self-Checking

Test bench
Input Signals
Output
Signals

Device Error/Status
Stimulus under Output
Checker Messages
Generator Verification
(DUV)

10
6 09/07/2013 Functional Verification
Any Q’s?

10
7 09/07/2013 Functional Verification

THANKS FOR YOUR PATIENCE & TIME

E-Mail ID:
[email protected]
[email protected]

10
8 09/07/2013 Functional Verification