CSC 431: Computer System Performance Evaluation (2 Units)

Introduction and overview, performance modelling; measurement techniques performance

monitoring, hardware monitoring, software monitoring, micro-coded instructions, statistical
techniques for computer performance analysis, statistical sampling for processor and cache
simulation, statistical simulation, benchmark simulation, simulation techniques; techniques,
workload characterization, performance evaluation in selection problems, performance
evaluation in design problems, evaluation of programme performance.

Lecture 1: Introduction and Overview of Computer System Performance Evaluation

Computer System Performance Evaluation is a specialized field focused on measuring,
analyzing, and optimizing the operational efficiency of computer systems. It encompasses a set
of methods and practices aimed at ensuring that systems deliver performance levels that meet
or exceed established requirements. Performance standards can vary significantly, depending
on the type of applications, the nature of the workload, user expectations, and specific system
constraints. Effective performance evaluation allows for the design, monitoring, and
refinement of systems that are not only responsive and efficient but also resilient to changing
demands.
Objectives of Performance Evaluation
1. Identifying Bottlenecks: Bottlenecks are components within a system that restrict
overall performance. These may include constraints in CPU processing power, memory
limitations, I/O capacity, network bandwidth, or other system resources. Bottleneck
identification is essential, as it enables targeted improvements that maximize system
throughput and responsiveness.
2. Optimizing Resource Allocation: Systems operate on finite resources, including
processing power, memory, storage, and network bandwidth. Efficient resource
allocation ensures these resources are used effectively to meet application needs and
workload demands without unnecessary waste.
3. Ensuring System Stability and Functionality: Performance evaluation ensures that
systems can handle various types and volumes of workloads while maintaining stability,
reliability, and responsiveness. This is critical, as performance degradation can result in
data loss, reduced productivity, and lower user satisfaction.
4. Balancing Trade-offs: Performance evaluation involves making trade-offs between
different metrics, such as balancing between high throughput and low response time, or
between high resource utilization and energy efficiency.
Importance of Performance Evaluation
In today's digital landscape, performance evaluation is indispensable for several reasons:
• System Reliability and Stability: Continuous performance evaluation ensures that the
system can handle increasing and varied workloads without crashing or significant
slowdowns, making it stable and dependable for users.

1
• Enhanced User Satisfaction: Efficient and responsive systems enhance the user
experience, ensuring quick load times, smooth interactions, and minimal delays, which
are vital for user retention and satisfaction.
• Cost Efficiency and Resource Conservation: Performance evaluation can reveal
opportunities for optimizing resource utilization, which reduces operational costs. By
identifying inefficiencies, organizations can minimize hardware investments and
energy consumption.
• Competitive Advantage: In sectors where service quality is directly impacted by
system performance (e.g., e-commerce, online banking), well-evaluated and optimized
systems can offer a competitive edge.
Performance Evaluation Techniques
Performance evaluation employs a variety of methods tailored to meet specific objectives. Key
techniques include:
1. Analytical Techniques: These techniques rely on mathematical models to analyze
system behavior, estimate performance metrics, and predict system response under
different conditions. Queueing theory is a commonly used analytical method, which
models processes such as CPU task handling and network packet processing to optimize
wait times and resource allocation.
2. Statistical Techniques: Statistical methods analyze data collected from the system to
identify trends, detect anomalies, and validate improvements. Statistical tools like
regression analysis, hypothesis testing, and sampling are used to assess system
performance comprehensively, allowing analysts to extrapolate findings to broader
scenarios.
3. Simulation and Modeling Approaches: Simulation allows for replicating system
behavior under controlled scenarios, providing valuable insights without impacting the
actual system. By simulating various workloads, resource configurations, and failure
scenarios, simulation techniques help forecast system behavior, enabling proactive
optimization. Modeling approaches, such as discrete event simulation, capture time-
sensitive events in system operation, providing a detailed understanding of performance
under real-world constraints.
Applications of Performance Evaluation
Performance evaluation has broad applications across different stages of system design,
implementation, and management. Key applications include:
• System Design and Prototyping: Before a new system design is finalized,
performance evaluation allows developers to predict and analyze how it will behave
under expected workloads. This early-stage analysis helps ensure that the design meets
performance requirements and avoids costly post-production modifications.
• Capacity Planning: Evaluating system capacity is crucial for organizations
experiencing growth. By assessing whether the system can scale up to meet future
demands, performance evaluation supports long-term planning and ensures resources
2
are provisioned effectively. This is particularly important for businesses anticipating
workload surges or expanding their operational scope.
• Performance Tuning and Optimization: For existing systems, performance
evaluation helps identify areas for improvement. This may involve refining process
scheduling, adjusting resource allocation, optimizing algorithms, or even upgrading
hardware. Tuning ensures that systems run efficiently, meeting performance standards
without unnecessary strain on resources.
• Security and Compliance: In many industries, systems must meet specific
performance and security standards. Performance evaluation ensures compliance with
industry standards (e.g., HIPAA for healthcare, PCI-DSS for payment processing) and
regulatory requirements, which often mandate specific levels of system performance
and security.
Key Performance Metrics
Performance metrics provide quantitative insights into system efficiency and are used to
measure, analyze, and compare system performance:
1. Throughput: Refers to the amount of work completed by a system within a given
timeframe. High throughput is often desirable, as it indicates the system's ability to
handle a large volume of tasks or transactions.
2. Latency: Measures the time delay experienced within the system, such as response time
between a user request and the system's response. Low latency is essential in
applications requiring real-time interactions.
3. Response Time: The time taken for a system to respond to a particular request or
command. Response time is especially critical in user-facing applications where delays
can directly affect user experience.
4. Utilization: The proportion of system resources actively used at any given time. High
utilization is often ideal, as it indicates that resources are well-leveraged, though
excessively high utilization can risk overloading and performance degradation.
5. Reliability and Uptime: Measures the system’s ability to operate continuously without
failure. Higher reliability is critical in systems where downtime could lead to significant
operational or financial losses.
Challenges in Performance Evaluation
Despite its importance, performance evaluation faces several challenges:
1. Complexity and Variability of Workloads: Modern systems handle diverse
workloads, each with different performance requirements. Evaluating systems across
varied conditions and balancing these requirements can be challenging.
2. Resource Limitations: In live environments, performance evaluation may be limited
by access to resources. For example, simulating high-stress scenarios could disrupt
regular operations, necessitating careful planning.

3
3. Accuracy of Models and Simulations: Simulations and models are approximations of
real-world scenarios. Ensuring these are accurate enough to provide reliable insights
can be difficult, especially in complex systems with many interdependencies.
4. Evolution of Technology: As technology evolves, new performance metrics may
become relevant, and existing methods may require adaptation. Staying current with
these changes can be challenging for organizations with limited resources.
Conclusion
Computer System Performance Evaluation is integral to creating and maintaining systems that
are not only functional and efficient but also reliable, stable, and scalable. It combines
theoretical and practical methods, from statistical analysis to advanced simulation techniques,
to deliver insights that guide system design, tuning, and optimization.
With continued advancements in hardware and software, performance evaluation techniques
must evolve, incorporating new metrics and approaches to meet the demands of modern
computing environments. Effective performance evaluation ensures that computer systems
meet today’s performance needs while remaining flexible enough to adapt to tomorrow’s
challenges.

Questions and Answers

Question 1: Identifying Bottlenecks at EU
Elizade University’s central server is facing slow response times during peak usage hours. How
can performance evaluation help identify bottlenecks in the system?
Answer:
Performance evaluation at EU can help identify bottlenecks by:
• Analyzing CPU, memory, and I/O utilization during peak hours.
• Using tools such as queueing theory to model system performance under varying
loads, and identifying if the CPU, memory, or network is under pressure.
• Monitoring real-time metrics like latency and response time to pinpoint the exact
bottleneck, whether it's in processing power, network bandwidth, or storage.
By identifying these bottlenecks, EU can optimize resource allocation and ensure the system
can handle peak loads more efficiently.

Question 2: Optimizing Resource Allocation at EU

EU's data center is experiencing inefficiencies in resource usage. How can performance
evaluation techniques be used to optimize resource allocation at EU?
Answer:
Performance evaluation techniques at EU can optimize resource allocation by:

4
• Implementing statistical analysis on resource usage data (e.g., CPU, memory, and
storage) to identify underutilized resources.
• Using simulation and modeling approaches to predict how different configurations
of resources (e.g., increased RAM or additional storage) impact overall performance.
• Ensuring that resources are distributed according to workload demands, using
techniques like dynamic resource scheduling to match computational power to peak
periods and optimize throughput.
This approach ensures that EU's data center operates at peak efficiency while minimizing
wasteful resource usage.

Question 3: Ensuring System Stability and Functionality at EU

During the end-of-semester exams, EU experiences heavy system usage. How can performance
evaluation ensure system stability during these periods?
Answer:
Performance evaluation can ensure system stability during high-demand periods by:
• Conducting capacity planning to assess if the current infrastructure can handle peak
demand (e.g., large numbers of students accessing the system simultaneously).
• Using simulation and modeling to simulate high traffic scenarios (e.g., thousands of
students accessing exam portals) and forecast how the system will behave under these
conditions.
• Monitoring reliability metrics to ensure uptime during critical times, and using load
balancing techniques to distribute the load evenly across multiple servers, preventing
system crashes.
By addressing potential weaknesses through these evaluations, EU can maintain stable
performance during critical periods.

Question 4: Enhancing User Satisfaction at EU

EU is aiming to improve the user experience for students accessing learning resources online.
How can performance evaluation help in enhancing user satisfaction?
Answer:
Performance evaluation can enhance user satisfaction by:
• Measuring response times to ensure that students experience minimal delays when
accessing course materials, videos, or submitting assignments.
• Monitoring throughput to ensure that the system can handle a large volume of requests
(e.g., during assignment submission deadlines).

5
• Using statistical techniques to analyze user feedback and performance data to identify
areas where delays or slow interactions occur, and adjusting system resources to
improve responsiveness.
Optimizing these metrics helps EU provide a faster, more reliable experience for students, thus
increasing overall user satisfaction.

Question 5: Addressing Security and Compliance Challenges at EU

EU is required to meet industry standards for data security and system performance. How can
performance evaluation ensure compliance with these standards?
Answer:
Performance evaluation ensures compliance by:
• Using performance metrics such as response time and uptime to ensure the system
meets service level agreements (SLAs) for reliability.
• Conducting security assessments alongside performance evaluations to ensure that the
system can handle security protocols without compromising system performance.
• Ensuring the system adheres to industry standards (e.g., GDPR for student data
protection) by evaluating system performance in terms of both efficiency and security
compliance.
Regular performance evaluation helps EU meet these standards and maintain the necessary
performance levels while ensuring data security.

Question 6: Calculating Throughput

A server can process 100 requests per second. How many requests can it process in 5 minutes?
Solution:
Throughput is measured in terms of requests per second.
• Requests per second = 100
• Time in seconds = 5 minutes = 5 * 60 = 300 seconds
The total requests processed = 100×300=30,000 requests.
The server can process 30,000 requests in 5 minutes.

Question 7: Latency Measurement

A system responds to a request in 50 milliseconds. How long would it take to process 1,000,000
requests sequentially?
Solution:
• Latency per request = 50 ms = 0.05 seconds
• Number of requests = 1,000,000
6
Total time = 0.05×1,000,000 = 50,000 seconds.
Convert seconds to hours: 50,000/3600 ≈ 13.89 hours.
It would take approximately 13.89 hours to process 1,000,000 requests sequentially.

Question 8: Response Time Calculation

A system has a latency of 30 milliseconds and the network delay is 15 milliseconds. What is
the total response time for a request?
Solution:
• Latency = 30 ms
• Network delay = 15 ms
Total response time = Latency + Network delay = 30 + 15 = 45 ms
The total response time for the request is 45 milliseconds.

Question 9: Utilization and Overload Risk

A server has 8 CPU cores, each capable of handling 100 tasks per second. The server is
currently handling 700 tasks per second. What is the current CPU utilization?
Solution:
• Total tasks handled by all cores = 8 × 100 = 800 tasks per second
• Current tasks being handled = 700 tasks per second
CPU utilization = 700/800 × 100 = 87.5%
The current CPU utilization is 87.5%.

Question 10: Reliability and Uptime

A system is designed to run for 365 days without failure, but it has experienced a total downtime
of 12 hours over the course of a year. What is the system’s reliability in terms of uptime
percentage?
Solution:
• Total downtime = 12 hours
• Total time in a year = 365 days = 365 × 24 = 8,760 hours
Uptime = Total time - Downtime = 8,760 – 12 = 8,748 hours
Reliability = 8,748/8,760 × 100 ≈ 99.86%
Answer:
The system’s reliability is approximately 99.86%.

7
Lecture 2: Performance Modeling in Computer System Evaluation
Performance modeling is the process of creating theoretical or empirical models that represent
the behavior of a computer system under various operating conditions. By developing these
models, system analysts can predict performance outcomes and identify potential bottlenecks
or areas for optimization. Performance models serve as invaluable tools in evaluating system
efficiency without the need for intensive testing on live systems, reducing costs and risks
associated with direct experimentation.
A. Types of Performance Models
Performance models can be categorized into several types, each with distinct characteristics
and applications:
1. Analytical Models
• Description: Analytical models use mathematical equations and representations to
simulate the relationships between different system components. These models are
generally simpler than other types but can provide quick, high-level insights into system
behavior.
• Example: Queueing models are a classic example of analytical models, often used to
predict system performance in environments where tasks are processed sequentially,
like CPU scheduling or network packet handling.
• Advantages: Analytical models are highly efficient and computationally inexpensive.
They provide general insights and can be applied across a wide range of scenarios.
• Limitations: These models often make simplifying assumptions that may not capture
the full complexity of real-world systems.
2. Simulation Models
• Description: Simulation models replicate the real-world operation of a system by
simulating its components and behaviors under controlled conditions. By mimicking
various system processes, these models allow for detailed exploration of different
scenarios, including those that are difficult to reproduce in a live environment.
• Example: Discrete event simulation models are commonly used to examine how
systems behave over time, tracking events such as job arrivals and departures from a
queue.
• Advantages: Simulation models offer flexibility and allow analysts to test various
"what-if" scenarios. They can accommodate complex interactions and provide detailed
insights into system dynamics.
• Limitations: These models can be computationally expensive and time-consuming,
especially for large or intricate systems. Accuracy is also dependent on the quality of
the simulation inputs.

8
3. Statistical Models
• Description: Statistical models rely on historical performance data to predict future
behavior. By analyzing trends and patterns, these models can generate probabilistic
forecasts that help in decision-making processes.
• Example: Regression analysis or machine learning models can be used to predict
workload performance, memory usage, or network traffic patterns based on past data.
• Advantages: Statistical models are effective in environments where data is abundant
and patterns are predictable. They can capture complex relationships that may not be
obvious.
• Limitations: These models depend heavily on the quality and quantity of data. They
may not perform well if data is insufficient or if the system behavior is highly variable
and unpredictable.
B. Applications of Performance Modeling
Performance modeling has a wide array of applications across different stages of system
development, optimization, and management:
1. System Design
• Purpose: During the design phase, performance modeling can be used to evaluate the
feasibility and efficiency of different system architectures. By modeling system
components and interactions, designers can predict how different configurations will
perform and select the most effective design.
• Example: In developing a new server configuration, performance modeling can predict
how different CPU, memory, and I/O combinations will handle the anticipated
workload. This helps designers build systems that meet performance requirements from
the outset.
• Benefits: Reduces the risk of costly design errors and allows for performance-focused
design decisions.
2. System Optimization
• Purpose: Once a system is operational, performance models help identify specific areas
where improvements can be made. This may involve enhancing processing speed,
reducing latency, or optimizing memory usage.
• Example: Modeling the CPU scheduling process can reveal inefficiencies in the current
algorithm, leading to refinements that reduce wait times for tasks and improve overall
throughput.
• Benefits: Optimization through performance modeling leads to resource efficiency and
improved user experience by ensuring the system operates at peak capacity.

9
3. Capacity Planning
• Purpose: Capacity planning involves predicting how well a system will handle future
workloads, particularly as demand grows or new applications are introduced. By
modeling performance under different load conditions, organizations can make
informed decisions about resource provisioning.
• Example: An e-commerce platform may use performance models to simulate peak
shopping periods, allowing the IT team to assess whether additional server capacity is
needed to handle increased traffic.
• Benefits: Prevents performance degradation due to resource constraints and ensures
that the system is scalable, resilient, and prepared for future demands.
Conclusion
Performance modeling is essential for developing efficient, reliable, and scalable computer
systems. By selecting appropriate modeling techniques—analytical, simulation, or statistical—
analysts can gain critical insights into system behavior, anticipate potential issues, and
proactively design solutions. Whether applied during design, optimization, or capacity
planning, performance modeling equips organizations to make data-driven decisions, ensuring
that their systems meet both current and future demands. Through rigorous performance
evaluation, organizations can maximize the value and longevity of their technological
investments, ultimately leading to greater operational success and user satisfaction.

Questions and Answers

Question 1: How can Analytical Models help EU in improving system performance?
EU's server system experiences performance issues, particularly during peak hours when many
students access the online learning portal. How can analytical models help resolve this?
Answer:
Analytical models, like queueing theory, can be applied at EU to analyze the behavior of the
server during high traffic. By using mathematical equations to model how tasks (such as student
requests for course materials) are processed sequentially, EU can predict how long tasks will
wait and how server components (e.g., CPU, memory) will respond under load. This model can
help identify potential bottlenecks and make simple, quick predictions about resource
allocation, such as how to adjust CPU or memory usage to better handle peak demand.
Benefits:
• Quick and computationally inexpensive.
• Provides insights into the capacity and limitations of current server resources.
• Helps to ensure that the system can handle anticipated peaks in workload without
delays.
Question 2: How can Simulation Models be used to test EU's online systems under
different conditions?

10
EU is planning to launch a new online examination platform. How can simulation models be
used to assess the system's performance?
Answer:
Simulation models, particularly discrete event simulation, can be used at EU to replicate the
behavior of the online examination platform under various conditions. This involves creating
a controlled environment where different scenarios (such as heavy user traffic during exam
periods) are simulated. The model can track events such as student logins, submission of
answers, and exam access, providing a detailed view of how the system will perform during
critical times.
Benefits:
• Flexibility to simulate multiple "what-if" scenarios.
• Allows EU to test performance under various conditions without affecting the live
system.
• Helps identify potential failures or bottlenecks, such as server overloads, before the
system goes live.

Question 3: How can Statistical Models assist EU in forecasting system performance

based on past data?
EU has collected performance data from its library systems over the past year. How can
statistical models help predict future performance and improve system efficiency?
Answer:
Statistical models, such as regression analysis or machine learning, can be applied to EU’s
historical performance data to predict future resource usage and performance metrics. By
analyzing patterns in past data, such as peak times when students access library resources, the
model can forecast CPU usage, memory consumption, and network traffic. This predictive
insight can help optimize resource allocation during high-demand periods, ensuring that
resources are efficiently distributed across EU’s systems.
Benefits:
• Predicts future performance trends based on historical data.
• Helps EU prepare for high-demand events, reducing system overload.
• Identifies patterns that may not be obvious, helping to optimize system resources.

Question 4: How can Performance Modeling assist EU in system design for future
expansion?
EU anticipates rapid growth in student numbers over the next few years. How can
performance modeling be used in the system design phase to ensure scalability?
Answer:
In the system design phase, performance modeling can help EU assess different system
configurations (such as adding more servers, improving CPU capacity, or increasing memory).
11
By simulating various architectures and their performance under predicted workloads (based
on expected student enrollment), EU can identify the best system configuration that meets the
needs of future growth. This ensures that EU’s infrastructure is both scalable and cost-efficient
from the outset.
Benefits:
• Helps design systems that can handle increasing demand over time.
• Provides a proactive approach to scaling resources, avoiding bottlenecks in the future.
• Reduces the risk of costly upgrades or redesigns after system deployment.

Question 5: How can Performance Modeling help EU optimize its IT infrastructure for
better user experience?
EU is facing user complaints about slow system performance during high-demand periods,
such as when students are registering for courses. How can performance modeling help
optimize system efficiency and improve user experience?
Answer:
Performance modeling can be used at EU to identify inefficiencies in how the system handles
peak loads. For example, using simulation models, EU can replicate high-demand periods like
course registration and test different strategies for resource allocation (such as adjusting server
load balancing or increasing memory). By identifying inefficiencies in CPU scheduling,
network traffic, or disk I/O, the system can be fine-tuned to ensure smoother performance
during peak times.
Benefits:
• Improves overall system efficiency, ensuring faster response times.
• Enhances the user experience by reducing delays and improving system availability.
• Ensures that the system operates at peak performance, especially during critical periods
such as registration and exam times.

12
Lecture 3: Performance Measurement Techniques
Performance measurement techniques are critical for understanding how well a computer
system operates under various conditions. These methods provide essential insights into system
behavior, efficiency, and areas that may require improvement. Proper performance
measurement ensures that a system meets performance standards, handles workloads
effectively, and performs optimally in its intended application.
A. Performance Metrics
Performance metrics provide measurable indicators that quantify various system
characteristics, allowing for a detailed evaluation of a system’s overall functionality. These
metrics can vary based on the specific system being evaluated, such as servers, networks,
databases, or user-facing applications. Here are some of the primary performance metrics:
1. Throughput
• Definition: Throughput is a measure of the volume of work a system can process over
a given time period. It is often used to measure the capacity of a system in handling
tasks like data processing, packet transmission, or user requests.
• Example: In a network, throughput might refer to the amount of data transmitted per
second (e.g., megabits per second, or Mbps), while in a database system, it could mean
the number of transactions or queries processed per second or minute.
• Significance: High throughput is indicative of a system's ability to handle significant
amounts of work efficiently, which is critical for systems dealing with large-scale data
processing or high traffic.
2. Latency
• Definition: Latency refers to the delay or time it takes to complete a task, process a
request, or deliver a response. It is a measure of the time interval between the initiation
of an operation and its execution or completion.
• Example: In a network context, latency could be the time it takes for a data packet to
travel from source to destination. For an application, it may represent the time it takes
for a user’s action (e.g., clicking a button) to be reflected in the system.
• Significance: Low latency is particularly important for real-time systems, such as
gaming, video conferencing, and financial trading platforms, where delays can
significantly impact user experience and system efficiency.
3. Response Time
• Definition: Response time is the total time elapsed from the initiation of a task or
request to its completion. This includes both latency and the time spent processing the
request or performing the task.
• Example: In a web application, response time is the time it takes from the moment a
user sends a request (e.g., clicking on a link) to the moment the page loads and is
displayed to the user.

13
• Significance: Short response times are critical for interactive applications where user
experience relies heavily on immediate feedback. Long response times can lead to user
frustration and poor system usability.
4. Utilization
• Definition: Utilization refers to the percentage of system resources actively in use at
any given time. This metric can apply to various system components, such as CPU,
memory, disk I/O, or network bandwidth.
• Example: CPU utilization might be measured as the percentage of the CPU’s capacity
used by active processes, while memory utilization refers to the percentage of available
RAM being used by running programs.
• Significance: High utilization typically indicates that the system is being used
efficiently, while very high or low utilization can indicate potential performance issues,
such as overloading or underutilization of system resources.
B. Measurement Tools and Techniques
Performance measurement tools and techniques are essential for gathering accurate, actionable
data on a system’s performance. These tools allow for continuous monitoring of system
operations, helping identify bottlenecks, diagnose issues, and ensure the system performs
within acceptable thresholds.
1. Performance Monitoring
• Description: Performance monitoring involves continuously or periodically collecting
data on various system performance parameters. This can include monitoring CPU
usage, memory consumption, disk I/O, network throughput, and response times.
• Types:
o Real-time Monitoring: Provides up-to-the-minute data on system
performance, allowing administrators to address issues immediately.
o Historical Monitoring: Collects data over a longer period to track trends,
identify recurring issues, and forecast future performance needs.
• Example: Tools like Windows Performance Monitor, Nagios, Zabbix, and
Prometheus allow IT teams to monitor a system's performance, providing real-time
alerts and historical data analysis.
2. Hardware Monitoring
• Description: Hardware monitoring utilizes physical counters and sensors embedded in
system hardware to measure low-level performance characteristics. This method
gathers data from hardware components like CPUs, memory, hard drives, and network
interfaces.
• Metrics Collected: Hardware monitoring can track things like CPU cycles, cache
misses, disk read/write operations, temperature, and power consumption.

14
• Example: Tools like Intel VTune, AMD Performance Profiler, and perf on Linux
access hardware counters to provide performance insights specific to CPU and memory
usage. These tools often allow for deep insights into system-level behavior, revealing
inefficiencies that software tools may overlook.
3. Software Monitoring
• Description: Software monitoring tools track system performance at the application or
process level. These tools record performance metrics such as disk usage, network
throughput, application-specific metrics, and memory consumption by processes or
threads.
• Example: New Relic and AppDynamics are widely used in application monitoring to
analyze transaction time, errors, response times, and throughput within software
systems, especially in web servers, databases, and cloud environments.
4. Micro-coded Instructions
• Description: Micro-coded instructions refer to low-level instructions embedded in
hardware to track and record system performance data with minimal overhead. These
instructions are highly efficient, collecting data directly from hardware components
such as the CPU and memory without significantly impacting system performance.
• Example: Intel Performance Monitoring Unit (PMU) and similar hardware on
modern CPUs use microcoded instructions to measure performance metrics such as
cycle counts, instruction cache misses, and branch prediction accuracy. These metrics
are collected without causing significant performance degradation, making microcoded
instructions valuable for high-frequency analysis.
C. Statistical Techniques for Performance Analysis
Statistical methods play a vital role in interpreting performance data, especially in complex
systems where patterns may be difficult to discern at first glance. These techniques help
identify trends, correlations, and anomalies that may not be immediately obvious.
1. Statistical Sampling
• Description: Statistical sampling involves collecting a subset of performance data
points at regular intervals or under controlled conditions. This allows for performance
analysis without overwhelming the system with constant measurement.
• Application: In CPU performance analysis, statistical sampling might be used to
collect data on instruction counts, memory accesses, and cache hits/misses at regular
intervals. These samples can then be used to infer system performance trends without
introducing significant overhead.
2. Statistical Simulation
• Description: Statistical simulation involves using statistical models to predict system
behavior under different conditions. By simulating various workloads and
environmental factors, this technique helps identify how system performance will scale
or change in response to various inputs.
15
• Example: Simulation models can predict how a new server setup will behave under
different user loads, helping in capacity planning and resource allocation decisions.
Conclusion
Performance measurement is an essential practice in understanding and optimizing computer
systems. By using the right metrics, such as throughput, latency, response time, and utilization,
alongside a combination of hardware and software monitoring tools, system performance can
be effectively assessed. Statistical techniques such as sampling and simulation provide further
depth to performance analysis, enabling proactive troubleshooting and optimization. Whether
through real-time monitoring, micro-coded instructions, or statistical models, accurate
performance measurement ensures systems operate efficiently, meet user demands, and
continue to evolve in response to growing workloads and new technological challenges.

Questions and Answers

Question 1: How can EU use Throughput as a performance metric to evaluate its network
system during peak hours?
During peak hours, many students at Elizade University (EU) access online resources, which
can overload the network. How can throughput help evaluate and optimize network
performance?
Answer:
Throughput measures the volume of data transmitted over the network in a given time frame.
At EU, it can be used to assess how much data is being transferred between servers and students'
devices, especially during busy periods like registration or exam times. High throughput
indicates that the network is capable of handling a large number of user requests and data
transfers efficiently, which is critical to avoid bottlenecks during peak usage.
Benefits:
• Helps identify if the network can handle peak traffic volumes.
• Provides insights into whether additional bandwidth is needed to improve performance
during busy periods.
• Helps avoid network congestion and slowdowns, ensuring that students can access
online resources without delay.

Question 2: How can Latency impact EU’s online exam system, and what can be done to
reduce it?
EU has an online exam system that students access remotely. Delays in response times may
affect exam performance. How does latency impact the system, and how can it be improved?
Answer:
Latency refers to the delay between sending a request (e.g., submitting an exam answer) and
receiving a response from the server. High latency in EU’s online exam system can lead to
16
delays in submitting answers or receiving updates, affecting students' experience, especially in
time-sensitive exams. To reduce latency, EU can optimize the server’s network infrastructure,
upgrade routers, and employ caching strategies to minimize delays in data retrieval.
Benefits:
• Reduces time delays during critical tasks such as exam submissions.
• Enhances the real-time interaction between the user and the system, improving the user
experience.
• Improves overall system responsiveness, ensuring a smooth experience during exams.

Question 3: How does Response Time affect EU's online learning portal, and how can it
be optimized?
Students at EU frequently report delays when accessing course materials on the online learning
portal. How does response time influence the user experience, and what strategies can be
employed to optimize it?
Answer:
Response time measures the time from when a student submits a request (e.g., clicking on a
course material link) until the page loads completely. Long response times can frustrate
students and hinder their ability to access materials efficiently. To optimize response time, EU
can improve the web server's performance, utilize content delivery networks (CDNs), optimize
database queries, and implement load balancing to distribute traffic more evenly across servers.
Benefits:
• Improves user experience by providing quick access to course materials.
• Enhances the accessibility and usability of the online portal, leading to higher student
satisfaction.
• Ensures that response times remain low, even under heavy usage, such as during exam
periods or course registration.

Question 4: What role does Utilization play in ensuring the efficiency of EU’s IT
infrastructure, and how can it be optimized?
EU's servers often experience performance degradation when many students are logged in
simultaneously. How does utilization impact server performance, and how can it be managed
effectively?
Answer:
Utilization refers to the percentage of system resources (such as CPU, memory, or network
bandwidth) actively used by the system. High utilization indicates that resources are being used
efficiently, but if utilization is too high, it can cause system overloads and slowdowns. To
optimize utilization, EU can monitor resource usage in real-time and allocate resources

17
dynamically based on demand, such as scaling server capacity during peak usage times or
upgrading hardware to handle larger workloads.
Benefits:
• Ensures that EU’s infrastructure is used efficiently without overloading resources.
• Prevents resource bottlenecks and system failures due to high utilization.
• Helps identify areas where hardware or software upgrades are needed to support
increased demand.

Question 5: How can EU leverage Performance Monitoring Tools to maintain system

performance and identify potential issues?
EU’s IT team struggles with identifying performance issues in real-time, which sometimes
leads to unaddressed bottlenecks. How can performance monitoring tools assist in
maintaining system performance?
Answer:
Performance monitoring tools allow EU’s IT team to collect real-time data on various system
parameters, such as CPU usage, memory consumption, network throughput, and response
times. Tools like Nagios, Prometheus, and Windows Performance Monitor can provide
alerts when thresholds are breached, allowing administrators to take immediate action.
Additionally, historical monitoring can help track performance trends over time, identify
recurring issues, and predict when hardware upgrades might be needed.
Benefits:
• Provides continuous insights into system performance, helping identify issues before
they become critical.
• Allows for proactive management of system resources, ensuring that bottlenecks are
addressed before they affect users.
• Helps optimize the system by tracking trends, preventing performance degradation over
time.

18
Lecture 4: Statistical Methods in Performance Evaluation
Statistical methods in performance evaluation offer a comprehensive approach to analyzing,
interpreting, and predicting system behavior based on empirical data. The application of
statistical techniques allows system administrators, engineers, and performance analysts to
make data-driven decisions, optimize system performance, and anticipate future challenges.
These methods enable a deeper understanding of system behavior, providing insights into how
a system behaves under varying workloads, operational conditions, and system configurations.

A. Key Statistical Techniques

1. Descriptive Statistics
o Definition: Descriptive statistics are vital for summarizing large volumes of
data and providing a simple yet comprehensive overview of system
performance. By using basic statistical metrics such as mean, median, mode,
standard deviation, and variance, analysts can capture the central tendency and
dispersion of performance data.
o Application: In computer system performance evaluation, descriptive statistics
are used to aggregate data points, such as response time, throughput, and
resource utilization. This can help determine if the system is operating within
acceptable limits or if certain performance aspects require optimization.
o Expanded Example: Consider a web application’s response time. An analyst
collects data on the response time for 1000 user requests. The mean gives the
average time taken for requests, the median shows the midpoint of response
times, the standard deviation reveals how much variation exists, and the
variance quantifies how much spread exists from the mean.
2. Inferential Statistics
o Definition: Inferential statistics allow analysts to make broader conclusions
about system behavior beyond the data at hand. By analyzing sample data,
inferential statistics help extrapolate how a system might behave under different
conditions or with additional load, making them crucial for predictive modeling.
o Application: Inferential statistics can be employed to predict the impact of
future system changes, such as how adding more RAM or changing the CPU
configuration will affect system performance. It also helps in identifying
whether the observed data supports specific hypotheses or if additional testing
is needed.
o Expanded Example: Suppose you want to predict system response time based
on various factors like CPU usage, memory load, and disk I/O. Using inferential
statistics, you can analyze sampled data from various system states and make
probabilistic predictions about system performance under future loads.

19
3. Regression Analysis
o Definition: Regression analysis examines the relationship between two or more
performance metrics, helping to determine how one metric impacts another.
This method is crucial for understanding the cause-and-effect relationships
between different aspects of system performance, such as CPU load and
response time, or memory usage and throughput.
o Application: Regression analysis can help in designing systems by predicting
how changes in one resource (e.g., increasing RAM) will affect other resources
(e.g., CPU utilization). It also helps in optimizing system configurations for
specific workloads, such as adjusting cache sizes to improve processor
performance.
o Expanded Example: You might use regression analysis to determine how CPU
utilization (independent variable) affects response time (dependent variable) in
a multi-tiered web application. The analysis might reveal that every 5% increase
in CPU utilization leads to a 2% increase in response time, helping to predict
performance during periods of high CPU load.
4. Hypothesis Testing
o Definition: Hypothesis testing involves testing a claim or assumption about
system performance to determine if observed differences are statistically
significant. This method helps to validate or reject performance improvement
strategies, new system configurations, or performance-enhancing features.
o Application: If a new memory allocation technique is proposed, hypothesis
testing can be used to compare the system's performance before and after
implementing the technique. If the difference in performance is statistically
significant, the new technique can be deemed effective.
o Expanded Example: Assume you hypothesize that upgrading the disk storage
from SSD to NVMe will reduce file access time. A hypothesis test could be
conducted by comparing the access times between the two storage
configurations. A t-test could be used to analyze if the reduction in access time
is statistically significant, providing evidence to support the upgrade.
B. Statistical Sampling (Expanded)
Statistical sampling is a technique for collecting a representative subset of performance data to
estimate overall system performance. It allows performance analysts to avoid the costs and
complexity of continuous system monitoring while still obtaining reliable data for evaluation
and optimization.
1. Processor and Cache Simulation
o Definition: Statistical methods are applied to simulate and model processor and
cache performance. This allows analysts to predict how changes in cache
configurations, memory hierarchies, or processor architectures will affect
overall system performance without the need for exhaustive testing.
20
o Application: Statistical simulation of processor and cache performance helps
determine the best configurations for minimizing cache misses, optimizing data
retrieval, and balancing memory access times. Analysts can explore different
cache designs, such as direct-mapped, set-associative, or fully-associative
cache, and predict their impact on system performance.
o Expanded Example: An analyst might simulate a processor’s cache
performance under different memory access patterns using statistical sampling.
The simulation might involve random access to memory and periodically
logging cache hit/miss ratios. By analyzing this data, the analyst can adjust
cache size and line replacement policies to maximize cache efficiency.
2. Random Sampling
o Definition: Random sampling involves taking measurements at random
intervals rather than at fixed intervals, ensuring that the data represents a broad
range of system states. This approach helps avoid bias and gives an overall view
of system performance across all usage scenarios.
o Application: Random sampling is useful in scenarios where constant
monitoring is impractical or where system behavior may vary unpredictably. By
sampling data at random intervals, it’s possible to identify performance trends
and anomalies without requiring full system instrumentation.
o Expanded Example: In a server system, random sampling might be used to
collect data on CPU load every 10-30 seconds. Over time, this random sample
will provide an estimate of overall CPU utilization across different load
conditions, from idle periods to peak traffic.
3. Stratified Sampling
o Definition: Stratified sampling divides the system’s operating conditions or
workload into distinct strata, ensuring that each stratum is adequately
represented in the sample. This technique improves the reliability of
performance data, especially in systems that exhibit different performance
characteristics under different workloads.
o Application: Stratified sampling is valuable for systems with highly variable
performance based on time of day, user load, or system configurations. By
collecting data from different strata (e.g., high, medium, and low load periods),
analysts ensure that all potential operating conditions are included in the
performance evaluation.
o Expanded Example: For a cloud-based application, stratified sampling might
involve collecting data during low, medium, and high traffic periods. This
ensures that performance evaluations take into account both normal and peak
usage scenarios, providing a more accurate picture of system performance under
different load conditions.

21
Conclusion
The application of statistical methods in computer system performance evaluation offers
powerful tools for understanding, predicting, and optimizing system performance. By using
descriptive statistics, inferential statistics, regression analysis, and hypothesis testing,
analysts can gain valuable insights into the factors that drive system performance and use these
insights to guide optimization efforts. Statistical sampling techniques such as random
sampling and stratified sampling provide a practical approach to data collection, enabling
performance analysis without overwhelming the system with constant monitoring. By
combining these techniques, performance engineers can ensure that systems operate efficiently,
meet user expectations, and scale appropriately as demands grow.

Questions and Answers

Question 1: How can Descriptive Statistics be used to assess system performance at Elizade
University’s online learning portal?

The online learning portal at Elizade University (EU) is experiencing fluctuating performance.
How can descriptive statistics help in understanding this variation?
Answer:
Descriptive statistics can summarize key performance metrics like response time, throughput,
and CPU utilization to provide a comprehensive view of system performance. By calculating
the mean, median, and standard deviation of response times across multiple user sessions, EU
can identify performance patterns and determine if the system is operating within acceptable
limits. For example, if the mean response time is high or the variance is large, it indicates
inconsistency in performance, suggesting areas that may require optimization.
Benefits:
• Identifies trends and outliers in system performance.
• Helps pinpoint whether system performance is stable or needs improvement.
• Offers a simplified overview for administrators to make quick decisions on
performance adjustments.

Question 2: How does Regression Analysis help Elizade University predict the impact of
increasing server capacity on user experience?
EU is considering adding more server capacity to improve the user experience on its online
exam system. How can regression analysis assist in predicting performance improvements?
Answer:
Regression analysis can be used to examine the relationship between server capacity (e.g.,
number of processors or RAM) and user experience metrics like response time or throughput.
By analyzing past data on CPU utilization and corresponding response times, regression
models can predict how increasing server resources will impact response times during peak
exam periods. For instance, if the regression model shows that increasing CPU capacity by
22
10% reduces response time by 5%, EU can estimate the improvements to the online exam
system’s performance when scaling up server resources.
Benefits:
• Provides a clear, data-driven prediction of how server upgrades will impact system
performance.
• Assists in making informed decisions on resource allocation for optimal performance.
• Reduces trial-and-error by predicting the results of infrastructure changes.

Question 3: In what way can Hypothesis Testing be used to validate a system optimization at
Elizade University?
EU is implementing a new memory management technique to optimize performance. How can
hypothesis testing help determine if the optimization is effective?
Answer:
Hypothesis testing can be used to compare system performance before and after the
optimization. For example, EU might hypothesize that the new memory management technique
will significantly reduce response time during peak usage. A t-test can be conducted to compare
the average response times from a sample of user requests before and after the implementation.
If the difference is statistically significant (i.e., the p-value is less than a predefined threshold),
the new technique can be deemed effective in improving performance.
Benefits:
• Provides statistical evidence on the effectiveness of optimization strategies.
• Helps eliminate biases or assumptions, ensuring decisions are based on data.
• Supports data-driven validation before further system-wide deployment.

Question 4: How can Random Sampling be used to monitor CPU load on EU’s server without
affecting performance?
EU’s IT team needs to monitor the CPU load on servers during high-demand periods but wants
to avoid overburdening the system with constant monitoring. How can random sampling be
applied?
Answer:
Random sampling involves collecting data at random intervals, which ensures that the data
represents a broad range of system states without overwhelming the server with continuous
monitoring. For example, EU could sample CPU load every 10-30 seconds at random times
during peak traffic. Over time, these random samples will provide an accurate estimate of CPU
utilization across various load conditions without significant performance degradation.
Benefits:
• Reduces the performance overhead of constant monitoring.
• Ensures a representative snapshot of system load without bias.
23
• Helps track CPU utilization trends and detect performance bottlenecks without real-
time monitoring.

Question 5: What is the role of Stratified Sampling in assessing Elizade University’s cloud-
based application performance during different traffic periods?
EU wants to assess the performance of its cloud-based application during different traffic
periods (e.g., low, medium, and high). How can stratified sampling help in this evaluation?
Answer:
Stratified sampling divides the system's performance data into distinct strata or segments based
on factors like traffic volume (low, medium, or high). By sampling performance data from each
stratum, EU ensures that performance evaluation accounts for all usage scenarios, from idle
periods to peak traffic. For example, stratified sampling during different traffic periods can help
identify how response times or throughput vary under varying load conditions, leading to more
accurate insights for capacity planning and optimization.
Benefits:
• Provides a more comprehensive and representative performance analysis.
• Accounts for variations in system behavior during different load levels.
• Improves decision-making by ensuring that performance data reflects all usage
scenarios.

Question 6: Hypothesis Testing – Testing the Impact of Upgrading Storage

Elizade University is considering upgrading the storage of its online portal from SSD to NVMe
to improve file access times. A sample of file access times (in seconds) before and after the
upgrade is shown below:
Before Upgrade (SSD): 0.8, 0.9, 1.0, 0.7, 0.85
After Upgrade (NVMe): 0.6, 0.55, 0.65, 0.6, 0.5

Question:

Test the hypothesis at a 5% significance level to determine if the upgrade significantly reduces
file access times.

24
Answer:
Based on the calculated t-value and comparison with the critical value, the hypothesis test will
provide a conclusion whether the file access times have been significantly reduced after
upgrading from SSD to NVMe. (For this, you would need to perform the calculations for t-
statistic and compare it to the critical value.)

25
Lecture 5: Benchmarking and Benchmark Simulation
Benchmarks in Performance Evaluation
Benchmarks are standardized tests designed to evaluate the performance of computer systems
under specific conditions. They provide valuable insights into the system's capabilities, offering
a means of comparison between different systems or configurations. Benchmarking is vital in
the design, optimization, and selection of systems, and it allows performance engineers to
assess improvements over time or after upgrades. The results from benchmarking can help
identify performance bottlenecks, determine system capabilities under various workloads, and
guide decisions regarding resource allocation and system configuration.
A. Types of Benchmarks
1. Synthetic Benchmarks
o Definition: Synthetic benchmarks use artificially generated workloads to test
specific aspects of a system's performance. These benchmarks focus on isolated
system components or functions, like the CPU, memory, or disk I/O. Synthetic
benchmarks are designed to stress a system component to determine its
maximum capacity or efficiency.
o Purpose: These benchmarks are primarily used to evaluate theoretical or raw
performance in controlled scenarios. They help in understanding the
fundamental capabilities of hardware or software components without
interference from other system processes.
o Example: Common synthetic benchmarks include the Linpack test for
evaluating CPU performance and memtest86 for assessing memory stability
and speed. Another example is the SPEC CPU benchmark, which tests
processor performance with a set of computational tasks.
o Advantages:
▪ Simple to set up and execute.
▪ Focus on a specific component, offering clear performance metrics.
▪ Useful for comparing systems or hardware configurations based on
isolated metrics.
o Limitations:
▪ May not reflect real-world performance in typical workloads.
▪ Performance may not scale well in practical, multi-component systems.
2. Application Benchmarks
o Definition: Application benchmarks evaluate system performance by running
real applications or workloads, simulating practical use cases such as database
queries, file transfers, or software compilation. These benchmarks measure how

26
well the system performs under conditions that closely resemble its expected
usage.
o Purpose: The goal of application benchmarks is to assess a system's
performance in real-world scenarios, considering factors like user interactions,
network latency, and application-specific demands.
o Example: Popular application benchmarks include the TPC-C benchmark for
database performance and SPECjbb for Java-based business applications.
Another common one is the WebBench benchmark for web server
performance.
o Advantages:
▪ Reflects true system performance as it applies to practical applications.
▪ Accounts for system behavior in multitasking environments.
o Limitations:
▪ Requires substantial setup and configuration, as the system must
replicate realistic workloads.
▪ Variability in workload types can affect benchmark consistency.
3. Microbenchmarks
o Definition: Microbenchmarks are designed to assess the performance of
individual system components, such as the CPU, memory, disk storage, or
networking devices. These tests focus on evaluating low-level system
performance, often at a granular level, and measuring the efficiency of specific
operations.
o Purpose: Microbenchmarks help in pinpointing performance issues in
individual subsystems, which can later be addressed to optimize overall system
performance.
o Example: Geekbench is a widely used microbenchmark that evaluates CPU
and memory performance. Disk performance benchmarks, such as HD Tune or
CrystalDiskMark, focus on measuring the speed and efficiency of hard drives
or SSDs.
o Advantages:
▪ Provides detailed information about specific system components.
▪ Useful for identifying performance bottlenecks in particular subsystems.
o Limitations:
▪ Does not reflect how the system performs under general workloads.
▪ Limited insight into system behavior when multiple components are
working together.

27
B. Benchmark Simulation Techniques
Benchmark simulations are vital tools in performance evaluation, allowing for the modeling
and testing of system performance without the need for extensive physical testing. These
techniques are particularly useful in simulating complex systems, optimizing configurations,
and understanding potential outcomes of system upgrades or modifications.
1. Trace-Driven Simulation
o Definition: Trace-driven simulation involves using recorded sequences of
events (traces) from actual workloads to simulate how a system will behave
under similar conditions. These traces represent the input/output behavior,
system calls, or network events that a real-world workload would generate.
o Purpose: The objective of trace-driven simulation is to recreate the real-world
behavior of a system, enabling a closer approximation of actual system
performance. By using traces, performance engineers can test a system's
responsiveness to real application workloads without executing them directly
on the system.
o Example: In database systems, traces might record SQL queries issued by users.
These traces can be played back through a simulator to measure system
performance under different loads or configurations, helping to evaluate
database optimizations or resource allocation strategies.
o Advantages:
▪ Allows for realistic performance evaluation based on real-world data.
▪ Enables testing of system performance in varied and complex use cases.
o Limitations:
▪ Requires access to comprehensive trace data.
▪ The simulation may not perfectly reflect every aspect of the real system,
especially in dynamic environments with unpredictable variables.
2. Event-Driven Simulation
o Definition: Event-driven simulation focuses on simulating specific events
within a system to model real-time behavior. In this technique, the simulation
runs in discrete steps, triggered by predefined events such as network requests,
disk accesses, or hardware interrupts.
o Purpose: Event-driven simulation is commonly used to model real-time
systems, network operations, and other scenarios where events occur
asynchronously. It helps in analyzing how a system responds to particular events
and the effect of these responses on system performance.
o Example: An event-driven simulation could model the behavior of a
networking system by simulating the arrival of data packets at various intervals,
allowing for the evaluation of packet loss, network congestion, and throughput.
28
o Advantages:
▪ Well-suited for modeling asynchronous or real-time systems.
▪ Can simulate complex systems and workloads with various types of
interactions.
o Limitations:
▪ Requires careful event definition and modeling.
▪ May involve high computational costs depending on the number of
events and the complexity of the simulation.
Conclusion
Benchmarking and simulation techniques are indispensable tools in computer system
performance evaluation. They provide a structured and objective means of assessing system
performance under various conditions, allowing for comparisons between systems,
configurations, and workloads. By using synthetic benchmarks, application benchmarks,
and microbenchmarks, performance engineers can focus on individual system components,
real-world applications, or theoretical performance. Moreover, trace-driven and event-driven
simulations offer powerful methods for simulating real-world behavior and evaluating how
systems respond to actual workloads or specific events. Combining these techniques enables a
holistic approach to performance analysis, optimization, and system design.

Questions and Answers

Question 1: Synthetic Benchmarks

Consider you are tasked with evaluating the performance of a new CPU that will be used in a
server farm. You decide to use synthetic benchmarks to perform the evaluation. Describe the
synthetic benchmarks you would use for this task, explaining why they are suitable for testing
the CPU's performance. Discuss one potential advantage and one limitation of using synthetic
benchmarks for this evaluation.
Answer:
• Benchmark Choice:
o Linpack: This benchmark evaluates the CPU’s performance by testing its ability
to perform floating-point operations, which are commonly used in scientific
computing and high-performance applications. Linpack is suitable for testing
raw processing power and how the CPU handles computationally intense tasks.
o SPEC CPU: This benchmark tests the processor’s performance with a variety of
computational tasks, simulating workloads such as data processing and
calculations that are relevant for server environments.
• Advantage:

29
o Clear Performance Metrics: Synthetic benchmarks focus on specific
components, offering simple, clear metrics that help compare the CPU’s
theoretical performance under controlled conditions.
• Limitation:
o Does Not Reflect Real-World Workloads: While synthetic benchmarks measure
the CPU’s maximum performance, they do not simulate typical server
environments where multiple components (e.g., memory, network, disk) are
interacting. Thus, they may not accurately represent real-world server
performance.

Question 2: Application Benchmarks

You are setting up a new e-commerce platform, and you want to ensure the system performs
well under realistic conditions. To do this, you decide to use application benchmarks. Identify
a suitable application benchmark for evaluating database performance in this scenario, explain
how it works, and discuss one advantage and one disadvantage of using application
benchmarks for evaluating the platform's performance.
Answer:
• Benchmark Choice:
o TPC-C Benchmark: This benchmark simulates a transaction-heavy workload,
which is typical for an e-commerce platform. It evaluates the system’s ability to
handle multiple concurrent transactions (e.g., product searches, checkout
processes, and inventory updates) and measures the system's throughput in a
transaction-processing environment.
• Advantage:
o Realistic Performance Measurement: Application benchmarks replicate real-
world usage, providing a more accurate assessment of how the system will
perform during actual operation, accounting for factors like network latency,
user interactions, and system multitasking.
• Limitation:
o Complex Setup: Application benchmarks often require significant setup and
configuration to mirror realistic workloads, which can be time-consuming and
resource-intensive, especially if the application environment is complex.

Question 3: Microbenchmarks
You are tasked with diagnosing performance bottlenecks in a high-performance storage system
used by a cloud service provider. You decide to use microbenchmarks to focus on individual
subsystems, such as the CPU and disk storage. Describe two microbenchmarks you would use
and explain how they would help you identify performance issues in this storage system. Also,
mention one advantage and one limitation of using microbenchmarks in this scenario.
Answer:
30
• Benchmark Choice:
o Geekbench: This microbenchmark evaluates CPU and memory performance,
which is useful for identifying whether the CPU is a limiting factor in the
system’s ability to process data quickly.
o CrystalDiskMark: This benchmark measures the read and write speeds of
storage devices (e.g., SSDs or HDDs), helping to identify any disk I/O
performance bottlenecks that could slow down data retrieval or storage.
• Advantage:
o Detailed Component Analysis: Microbenchmarks provide detailed insights into
the performance of individual components (CPU, memory, disk), which is
crucial for pinpointing specific bottlenecks in the system.
• Limitation:
o Limited System Context: Microbenchmarks do not account for how multiple
components interact under real-world workloads. For example, disk
performance might be fine under isolated tests but could become a bottleneck
when combined with other system components in a full workload.

Question 4: Trace-Driven Simulation

You are developing a simulation to assess the potential impact of increasing the number of
users accessing a web application in a cloud environment. You decide to use trace-driven
simulation to model real-world user interactions. Explain how trace-driven simulation works
and describe one advantage and one limitation of this simulation technique in the context of
modeling the web application's performance.
Answer:
• Simulation Explanation:
o Trace-Driven Simulation involves using recorded sequences of user interactions
(e.g., web page requests, database queries) to simulate the behavior of the
system under real-world conditions. By playing back these traces through the
simulator, you can evaluate how the system responds to a higher load of user
requests and measure key performance metrics like response time, throughput,
and resource utilization.
• Advantage:
o Realistic Workload Simulation: Trace-driven simulation allows you to test the
system’s performance based on real user behavior, providing more accurate
insights into how the system will perform under actual usage conditions.
• Limitation:
o Dependence on Trace Data: The accuracy of the simulation depends on the
availability and completeness of trace data. If the recorded traces do not fully
represent the diversity of user interactions or changing usage patterns, the

31
simulation may not perfectly reflect the system's performance under new or
unforeseen conditions.

Question 5: Event-Driven Simulation

In a networked system where data packets are transmitted in real-time, you decide to use event-
driven simulation to model the impact of network congestion on system performance. Describe
how event-driven simulation works and discuss one advantage and one limitation of using this
simulation method for evaluating real-time network performance.
Answer:
• Simulation Explanation:
o Event-Driven Simulation models the system by simulating discrete events, such
as the arrival of data packets or network congestion events. The system evolves
in response to these events, and the simulation tracks the system’s behavior as
it processes each event. In a network simulation, you might model the
transmission of packets between nodes, measure latency, and evaluate how
congestion impacts data throughput and packet loss.
• Advantage:
o Modeling Asynchronous Events: Event-driven simulation is ideal for modeling
systems with asynchronous or real-time behavior, where events like data
transmission or network congestion occur unpredictably and need to be handled
as discrete events.
• Limitation:
o High Computational Cost: Event-driven simulations can be computationally
intensive, especially when modeling large-scale networks or complex systems
with many events. The simulation’s complexity increases with the number of
events, which can require significant computational resources to run efficiently.

Question 6: Benchmark Simulation – Event-Driven Simulation for Network Throughput

A network system is simulated using event-driven simulation to evaluate its throughput. The
system generates data packets at intervals, and the following events (in milliseconds) are
recorded for 5 packets:

32
Question
Calculate the average packet arrival time and throughput of the network system.
Solution:
1. Average Packet Arrival Time Calculation:
The formula for the average packet arrival time is:

2. Throughput Calculation:
Throughput is the inverse of the average packet arrival time. If the system generates
packets at a rate, the throughput in packets per second (pps) can be calculated as:

Answer:
• Average Packet Arrival Time: 13.8 ms
• Throughput: 72.46 packets per second

33
Lecture 6: Workload Characterization in Performance Evaluation
Workload characterization remains one of the foundational activities in system performance
evaluation. It provides insights into the nature of the tasks the system performs, how resources
are consumed during these tasks, and how these behaviors change over time or under varying
conditions. Properly characterizing workloads enables system designers, administrators, and
performance engineers to optimize the system for different use cases, leading to better
performance, efficiency, and scalability.
A. Characteristics of Workloads
Workload characterization revolves around understanding the intensity and the specific
requirements of the resources used during system operation. A single application or task may
not be fully representative of a system's performance, but collectively, the various types of
workloads that a system handle can inform important system design and optimization
decisions.
1. I/O Bound Workloads
o Definition: I/O-bound workloads are tasks that are limited by the system’s
input/output operations rather than CPU processing. In I/O-bound workloads,
the system spends more time waiting for data to be read from or written to
storage devices (like hard drives, SSDs) or network resources, rather than
performing computations.
o Example: A typical example would be a database server that regularly
retrieves or stores large datasets from disk or a file server handling a continuous
stream of file read/write requests. Another example is web servers that serve
static content (images, files, etc.) from disk to users over the network.
o Performance Implications:
▪ These workloads often experience significant delays due to disk access
times or network congestion.
▪ Improving the throughput of the I/O system is critical. This could
involve using faster storage devices (e.g., SSDs) or optimizing the
network layer (e.g., increasing bandwidth, reducing latency).
▪ Techniques like disk caching, RAID configurations, and load
balancing can mitigate the effects of I/O bottlenecks.
2. CPU Bound Workloads
o Definition: CPU-bound workloads are tasks that are predominantly limited by
the processing power of the central processing unit (CPU). These workloads
require significant computation and data processing capabilities, with CPU
usage being the primary factor affecting performance.
o Example: Scientific simulations, video encoding, and cryptographic
calculations are typical CPU-bound workloads. Applications that perform

34
extensive numerical computations, such as machine learning algorithms or big
data processing, often fall into this category.
o Performance Implications:
▪ These tasks benefit from a high-performance CPU, typically one with
multiple cores or processors, to parallelize workloads.
▪ Cache memory optimization is critical, as the performance of CPU-
bound workloads is often limited by the ability to fetch instructions and
data from memory quickly.
▪ Multi-core processors can be leveraged to execute multiple tasks
simultaneously, reducing time for computation-heavy processes.
3. Memory Bound Workloads
o Definition: Memory-bound workloads occur when system performance is
constrained by the availability or access speed of the system’s memory (RAM).
In these workloads, the system has to deal with large data sets that require
significant memory resources and frequent memory accesses.
o Example: Examples of memory-bound workloads include data mining
applications, in-memory databases like Redis, or large-scale simulations
where large arrays or data structures must be manipulated in memory.
o Performance Implications:
▪ Efficient memory management is critical. Systems need to ensure that
they are making the best use of their available memory and managing it
in a way that minimizes latency (e.g., through efficient paging and
segmentation).
▪ Strategies like memory paging, virtual memory, and cache
management can help mitigate issues that arise from large data sets.
▪ Adding more RAM or using memory optimizations (e.g., larger cache
sizes) can improve performance by reducing bottlenecks in memory
access.
4. Network Bound Workloads
o Definition: Network-bound workloads are limited by the system’s ability to
transmit data over a network. These tasks heavily rely on the speed, bandwidth,
and latency of the network, and the system’s performance is hindered if the
network is slow or congested.
o Example: Streaming services (e.g., Netflix, YouTube), cloud applications
(e.g., Google Drive, Dropbox), and real-time communication systems (e.g.,
video conferencing) are examples of network-bound workloads, where the
system must send and receive large amounts of data over a network.

35
o Performance Implications:
▪ Network bandwidth and latency are crucial performance factors in
network-bound workloads. Increasing bandwidth, reducing congestion,
and optimizing routes for data transfer are key performance
improvements.
▪ For high-throughput applications like data centers, efficient load
balancing, quality of service (QoS), and caching can help ensure
consistent network performance.
▪ In distributed systems, optimizing network protocols and ensuring
minimal data transfer between nodes can lead to performance gains.
B. Techniques for Workload Characterization
Properly characterizing workloads involves gathering data about resource consumption and
analyzing the results to identify patterns and inefficiencies. Several techniques are available to
evaluate workloads, from simple profiling to sophisticated data analysis and prediction models.
1. Profiling
o Definition: Profiling is the process of collecting detailed data on how a system
performs under different workload types. Profiling helps monitor the
consumption of system resources like CPU, memory, disk I/O, and network
usage.
o Purpose: Profiling allows you to pinpoint performance bottlenecks and
underutilized resources, enabling targeted optimizations. By understanding how
workloads interact with system resources, engineers can make better decisions
about hardware upgrades and software optimizations.
o Tools and Techniques: Tools like gprof, strace, perf, and vmstat can capture
system performance data, including CPU cycles, memory usage, and I/O
operations during the execution of a workload.
2. Classifying Workloads
o Definition: Classifying workloads involves categorizing different types of
workloads based on their behavior and resource usage patterns. This
classification helps system designers understand the system’s needs under
varying conditions.
o Purpose: By classifying workloads, system architects can better allocate
resources and optimize configurations. This is particularly useful in shared
environments or when planning for system scalability.
o Methodology: Workloads can be classified based on:
▪ Resource intensity (I/O, CPU, memory, or network).
▪ Frequency (e.g., how often tasks are repeated).

36
▪ Peak load (e.g., workloads that experience spikes).
▪ Operational time (e.g., workloads that run continuously or
intermittently).
3. Trends Analysis
o Definition: Trends analysis involves examining historical data to identify
patterns in system performance and predict future resource needs. By analyzing
past workload behavior, engineers can anticipate future performance
requirements and allocate resources accordingly.
o Purpose: Trends analysis helps in capacity planning and ensures that the system
can handle future workloads efficiently. It also allows for the detection of
potential performance degradation over time.
o Tools: Tools like R, Python, and Excel are commonly used for visualizing data
trends and making forecasts based on historical patterns. Predictive models and
machine learning algorithms can also be used to predict future system demands.
Conclusion
Effective workload characterization is a critical aspect of performance evaluation in computer
systems. Understanding the nature of the workloads a system handles—whether I/O-bound,
CPU-bound, memory-bound, or network-bound—provides invaluable insights that guide
system optimization efforts. By applying techniques such as profiling, classification, and trends
analysis, system designers and performance engineers can gain a deep understanding of the
system's behavior and make informed decisions about resource allocation, system tuning, and
future scalability. Proper workload characterization helps ensure that a system can perform
optimally under varying conditions and meet the ever-evolving demands of users and
applications.

Questions and Answers

Question 1
Explain the four main types of workloads, providing an example of each that could be
relevant to the IT infrastructure at Elizade University (EU).
Answer:
1. I/O Bound Workloads
o Definition: Tasks constrained by input/output operations rather than CPU
processing.
o Example (EU): The university's library database system retrieving large
datasets for staff and students during peak usage periods.
2. CPU Bound Workloads

37
o Definition: Tasks requiring significant computational power, limited by CPU
speed.
o Example (EU): Scientific research simulations conducted by students or faculty
in the engineering department.
3. Memory Bound Workloads
o Definition: Tasks limited by the speed or availability of RAM.
o Example (EU): Processing large datasets for student enrollment and record
management during admission periods.
4. Network Bound Workloads
o Definition: Tasks constrained by the speed and latency of the network.
o Example (EU): Real-time video streaming during online lectures or virtual
meetings on platforms like Zoom.

Question 2
Discuss two performance implications of I/O-bound workloads and suggest ways EU can
optimize its systems to handle these workloads efficiently.
Answer:
1. Performance Implications:
o Delays due to slow disk access or network congestion.
o Limited throughput affecting user experience, especially during peak access
periods.
2. Optimization Strategies:
o Upgrade storage systems to SSDs for faster data retrieval.
o Implement load balancing on the library's file servers to distribute requests
evenly during high traffic times.

Question 3
What are the main goals of profiling in workload characterization, and how could EU use
profiling to enhance its student information system?
Answer:
1. Goals of Profiling:
o Identify performance bottlenecks and underutilized resources.
o Gather detailed data about system resource consumption during workload
execution.
2. Application at EU:
38
o Use tools like perf or vmstat to monitor CPU and memory usage of the student
information system.
o Address any identified bottlenecks, such as high memory usage during exam
result uploads, by optimizing the underlying application code or database
queries.

Question 4
Why is classifying workloads important, and how can EU benefit from this technique in
managing its IT resources?
Answer:
• Importance:
o Enables better resource allocation and performance optimization.
o Facilitates system scalability and planning for peak demand times.
• Benefits for EU:
o Classify workloads based on intensity (e.g., high I/O for library systems vs. high
CPU for research projects).
o Allocate dedicated servers for high-frequency workloads like the university
portal to ensure continuous availability during student registration.

Question 5
How can trends analysis in workload characterization support capacity planning for EU’s
network infrastructure?
Answer:
• Role of Trends Analysis:
o Identifies patterns in system usage over time, allowing prediction of future
resource needs.
o Helps in preemptive upgrades or adjustments to avoid performance degradation.
• Application for EU:
o Use historical data from online learning platforms to anticipate bandwidth
spikes during examination weeks and allocate additional network capacity.
o Predict growth in user numbers and scale up Wi-Fi access points in student
hostels to support increased traffic.

39
Question 6: Elizade University's online portal server handles 10,000 requests daily. Each
request requires an average of 5 milliseconds of CPU processing time and 50 milliseconds of
I/O operations. Calculate the daily CPU and I/O utilization in seconds.
Solution:
• CPU Utilization per request = 5 ms = 0.005
• I/O Utilization per request = 50 ms = 0.05 seconds
• Total Daily CPU Time = 10,000 × 0.005 = 50 seconds
• Total Daily I/O Time = 10,000 × 0.05 = 500 seconds
Answer:
The server uses 50 seconds of CPU time and 500 seconds of I/O time daily.

Question 7: A virtual lecture at EU streams at an average bitrate of 2 Mbps. If 500 students

attend the lecture simultaneously, calculate the total bandwidth required for the session in Gbps.
Solution:
• Bandwidth per user = 2 Mbps
• Total Bandwidth = 500 × 2 Mbps = 1000
• Convert Mbps to Gbps:
1000 Mbps = 1000/1000 Gbps = 1 Gbps
Answer:
The total bandwidth required is 1 Gbps.

Question 8: A memory-bound workload at EU requires 2,000 memory accesses per task, with
each access taking 100 nanoseconds. After optimization, memory access time is reduced to 80
nanoseconds. Calculate the percentage reduction in memory access time and the total time
saved per task.
Solution:

40
Lecture 7: Performance Evaluation in System Selection and Design
Performance evaluation is a critical process in both the selection and design of computer
systems. It ensures that the system chosen or designed meets the operational requirements,
performs optimally under expected workloads, and provides scalability for future growth.
Below is a detailed, extended breakdown of how performance evaluation applies to system
selection and design.

A. Performance Evaluation in System Selection

When selecting a computer system, the primary objective is to choose a system that aligns with
the specific requirements of the application, user demands, and the organization’s goals. The
process of system selection involves evaluating the potential systems based on their
performance characteristics, scalability, and cost-effectiveness.

1. Comparative Analysis
Purpose: Comparative analysis is fundamental to the decision-making process. By comparing
the performance characteristics of different systems, organizations can determine the most
efficient solution that balances performance, cost, and future requirements.
Process:
• Identifying Metrics: Before comparison, establish the key performance metrics such
as throughput, response time, CPU utilization, and memory bandwidth, which are
critical for the specific use case.
• Cost-Benefit Assessment: Perform a cost-benefit analysis that compares initial system
costs (e.g., hardware, software) and long-term operational costs (e.g., maintenance,
energy consumption, upgrade costs).
• Application Scenarios: Simulate or benchmark the system in actual workloads to
understand its real-world performance and validate vendor claims.
• Scenario Planning: Anticipate future growth, such as increasing user load or data
volume, and analyze which system will handle such scenarios efficiently without
incurring prohibitive costs.
Example: A comparison between traditional on-premises hardware solutions and cloud-based
infrastructures like AWS or Azure, weighing performance against cost, reliability, and
scalability for specific workloads like high-volume data processing or web hosting.

2. Compatibility Testing
Purpose: Compatibility testing ensures that the selected system will work effectively with the
existing software, hardware, and network environments. This is especially important for legacy
systems or highly specialized environments.

41
Process:
• System Integration: Test the system in the context of the organization’s existing
hardware, software, and network infrastructure.
• Software Compatibility: Ensure that critical applications run smoothly on the new
system, considering dependencies on operating systems, specific drivers, or libraries.
• Interoperability Testing: Evaluate how the selected system interacts with other
systems, such as third-party services, peripheral devices, or data storage solutions.
• Performance with Legacy Systems: Test how the new system will perform when
interacting with legacy systems, ensuring minimal disruption.
Example: In selecting a new database server, you would need to verify its compatibility with
your organization’s existing enterprise resource planning (ERP) system or customer
relationship management (CRM) software.

3. Reliability Testing
Purpose: Reliability testing ensures that the system performs consistently under typical and
peak workloads without failures or crashes, which is essential for mission-critical applications.
Process:
• Stress Testing: Simulate peak load conditions to identify how the system behaves under
high-stress scenarios.
• Fault Tolerance Testing: Assess the system’s ability to recover from hardware failures,
software crashes, or power outages.
• Uptime Testing: Evaluate the system’s uptime and measure failure rates over extended
periods of continuous use.
• Redundancy and Backup: Ensure that mechanisms for redundancy (e.g., RAID
arrays, failover systems) are working properly and providing system availability in case
of component failures.
Example: When selecting a cloud service, you would assess the service level agreement (SLA)
to guarantee that it offers adequate uptime and fault tolerance for your critical services, such as
email hosting or e-commerce platforms.

B. Performance Evaluation in System Design

In system design, performance evaluation ensures that the system architecture and its
components will meet the performance targets set forth during the planning stages. Design-
phase evaluation focuses on testing the initial prototypes, analyzing scalability, and optimizing
key system components.

42
1. Design Prototyping
Purpose: Prototyping allows system designers to test early versions of the system before full-
scale deployment, identifying potential performance bottlenecks, scalability issues, or
integration challenges.
Process:
• Prototype Development: Develop a basic version of the system, often with reduced
functionality, to quickly assess its performance under controlled conditions.
• Iterative Testing: Run the prototype through different workload scenarios, fine-tuning
the design with each iteration based on performance data.
• Early Feedback: Collect feedback from end-users or stakeholders regarding
performance metrics such as usability, responsiveness, and reliability.
• Hardware and Software Integration: Test how well different hardware and software
components work together, particularly in systems with complex architectures (e.g.,
distributed or cloud-based systems).
Example: In developing a new e-commerce platform, you would create a basic prototype to
simulate transactions, user load, and payment processing under real-world conditions.
2. Scalability Testing
Purpose: Scalability testing ensures that the system can handle increasing workloads and user
demands over time without degradation in performance. It is crucial for systems expected to
grow in size or complexity.
Process:
• Load Testing: Gradually increase the number of simultaneous users or transactions to
measure how the system’s performance changes with increased demand.
• Vertical Scalability: Assess the system's ability to scale by adding more resources (e.g.,
CPU, RAM) to the existing system, such as upgrading a server or increasing memory
capacity.
• Horizontal Scalability: Test the system’s ability to scale horizontally, by adding more
machines or nodes (e.g., adding more web servers to a load-balanced environment).
• Capacity Planning: Predict future growth based on current performance data and
define scaling strategies, ensuring that additional resources will maintain system
efficiency.
Example: In designing a cloud service, scalability testing would ensure that the platform can
accommodate increasing user demand without performance degradation, utilizing autoscaling
features in cloud environments like AWS or Google Cloud.

43
3. Optimization
Purpose: Optimization aims to enhance system performance by addressing inefficiencies
identified during testing and ensuring that the system performs at its best under varying
workloads.
Process:
• Bottleneck Identification: Analyze performance data to identify critical areas where
the system is underperforming, such as memory usage, I/O throughput, or network
latency.
• Resource Allocation: Optimize how resources (e.g., CPU, memory, storage) are
allocated, ensuring that critical processes receive sufficient resources to operate
effectively.
• Algorithmic Optimization: Improve the algorithms used within the system, such as
optimizing sorting or search algorithms to reduce CPU load or improving network
protocols to decrease latency.
• Load Balancing: Implement or refine load balancing strategies to distribute workloads
evenly across resources, reducing the risk of overload and improving response times.
Example: In optimizing a content delivery network (CDN), you would focus on improving
content caching algorithms, minimizing latency, and ensuring that content is served efficiently
to users based on their geographic location.

Conclusion
Performance evaluation in system selection and system design ensures that computer systems
meet performance requirements and provide reliable, scalable, and cost-effective solutions for
users. In system selection, the process focuses on comparing systems, ensuring compatibility,
and testing reliability to choose the best system for the organization. In system design,
performance evaluation helps refine prototypes, assess scalability, and optimize system
components to deliver a high-performance system under real-world conditions.
By employing these detailed performance evaluation techniques, organizations can ensure that
the systems they select or design will meet the performance expectations of their users while
accommodating future growth and increasing demands. Performance evaluation thus provides
a framework for decision-making and optimization that is vital to maintaining efficiency,
competitiveness, and user satisfaction in dynamic technological environments.

44
Questions and Answers

Question 1
Explain how Elizade University (EU) can use comparative analysis to select the most
suitable server infrastructure for its online student portal.
Answer:
• Identifying Metrics: EU should evaluate metrics such as server throughput, response
time, and CPU utilization.
• Cost-Benefit Assessment: Compare on-premises hardware costs versus cloud hosting
solutions like AWS or Azure, factoring in maintenance and scalability.
• Application Scenarios: Benchmark server performance under typical workloads, such
as student registrations and result uploads.
• Scenario Planning: Consider future growth, such as increased student enrollment, to
ensure scalability.
• Example: Compare traditional dedicated servers with cloud-based infrastructure to
decide which best handles peak workloads during exam results release.

Question 2
Describe the process and importance of compatibility testing in implementing a new
Learning Management System (LMS) at EU.
Answer:
• System Integration: Test the new LMS with EU’s existing hardware and software
infrastructure, including current authentication systems.
• Software Compatibility: Ensure the LMS runs smoothly on EU’s operating systems
and integrates with existing tools like Office 365.
• Interoperability Testing: Evaluate interaction with third-party services, like online
testing platforms or video conferencing tools.
• Legacy System Performance: Test compatibility with older systems, such as student
databases, to prevent disruptions.
• Importance: Ensures seamless deployment, reduces downtime, and avoids costly
rework after implementation.

Question 3
Using a practical example at EU, explain how stress testing can improve the reliability of
the university’s IT infrastructure.
Answer:

45
• Process: Simulate peak conditions, such as thousands of simultaneous logins during
registration.
• Fault Tolerance: Evaluate how the system recovers from unexpected server failures or
database crashes.
• Redundancy Testing: Verify failover mechanisms, such as backup servers, to ensure
continuity during hardware failures.
• Example: During stress testing of EU’s student portal, simulate 10,000 concurrent
users attempting to register, identifying bottlenecks and optimizing server resources to
prevent crashes.
• Outcome: Improved reliability ensures that critical services remain available even
under heavy usage.

Question 4
How can EU use scalability testing to ensure its IT systems can handle future growth in
student population?
Answer:
• Load Testing: Gradually increase simulated user numbers to measure system response
times and throughput.
• Vertical Scalability: Test performance when adding resources, such as more RAM or
faster CPUs, to existing servers.
• Horizontal Scalability: Evaluate the ability to add more servers, ensuring seamless
integration into the network.
• Capacity Planning: Predict growth trends, such as a 20% annual increase in student
enrollments, and plan upgrades accordingly.
• Example: Test EU’s e-learning platform to accommodate a projected increase from
5,000 to 10,000 users while maintaining smooth performance.

Question 5
Identify two optimization techniques EU could use to improve its campus network
performance and provide examples of their application.
Answer:
1. Bottleneck Identification: Use performance monitoring tools to detect high network
latency during online lectures.
o Application: Upgrade switches or optimize routing protocols to reduce delays
during live streaming of classes.
2. Load Balancing: Implement load balancers to distribute traffic evenly across multiple
servers.

46
o Application: Use load balancing during peak periods, such as exam
registrations, to prevent server overload and improve response times.

Question 6
EU is comparing on-premises servers and cloud hosting for its online portal.
• On-Premises Costs:
o Initial hardware setup: ₦10,000,000
o Maintenance: ₦1,000,000/year
o Lifespan: 5 years
• Cloud Hosting Costs:
o Monthly subscription: ₦200,000
Calculate the total cost for each option over 5 years and determine the cost difference.

Solution:
1. On-Premises Cost:
o Total = Initial setup + (Maintenance × 5 years)
o Total = ₦10,000,000 + (₦1,000,000 × 5) = ₦15,000,000
2. Cloud Hosting Cost:
o Total = Monthly cost × 12 months × 5 years
o Total = ₦200,000 × 12 × 5 = ₦12,000,000
3. Cost Difference:
o Difference = ₦15,000,000 - ₦12,000,000 = ₦3,000,000
Answer:
Over 5 years, cloud hosting is ₦3,000,000 cheaper than on-premises servers.

Question 7
EU’s student portal processes 5,000 requests per hour during peak registration.
• Each request takes an average of 300 milliseconds (ms) to complete.
• The server can handle requests in parallel using 10 cores.
Calculate the average throughput (requests per second).
Solution:
1. Time per Request (seconds):
o 300 ms = 300/1000 s = 0.3 s
2. Requests per Core per Second:

47
o 1/0.3 = 3.33 requests/second
3. Total Throughput:
o 3.33 × 10 = 33.3 requests/second
Answer:
The system has an average throughput of 33.3 requests per second.

48
Lecture 8: Evaluating Program Performance
Evaluating program performance is an essential aspect of software development that aims to
ensure efficient resource usage and meet performance requirements. This evaluation helps in
identifying performance bottlenecks, understanding resource consumption, and ensuring that
the program performs optimally in various execution environments. Effective performance
evaluation also allows developers to optimize code, enhance user experience, and ensure that
the program meets its functionality and efficiency goals.

A. Program Profiling
Definition: Program profiling refers to the process of collecting runtime data about a program
to analyze its behavior and identify inefficiencies. Profiling helps developers understand where
the program spends most of its time, which parts of the code consume significant resources,
and how to optimize those sections to improve overall performance.
1. Types of Profiling Tools
• Time Profilers: Measure the execution time of individual functions, methods, or code
blocks to identify performance bottlenecks. These tools help in detecting which parts
of the program are slow and may require optimization.
• Memory Profilers: Track memory usage during execution, identifying memory leaks,
excessive memory consumption, or inefficient memory allocation. These tools can
highlight areas where memory optimization is necessary.
• I/O Profilers: Track the efficiency of input/output operations in the program, such as
file reading, disk access, or network communication. Identifying I/O bottlenecks helps
optimize data handling and transfer mechanisms.
• Concurrency Profilers: Measure how effectively the program handles concurrent
execution, including threads, processes, and synchronization. Concurrency profiling
can reveal issues such as thread contention, excessive locking, and deadlocks that can
impact performance.
2. Profiling Process
• Data Collection: Profilers collect various performance-related data, such as time spent
in functions, memory allocations, and disk or network activity. The data collected is
typically used for subsequent analysis and optimization.
• Analysis: Profiling results are analyzed to identify performance bottlenecks, resource
consumption patterns, and inefficient code paths. Tools like flame graphs, heatmaps,
and performance graphs are often used to visualize performance issues.
• Optimization: After analyzing the data, developers can optimize critical sections of
code. Optimization techniques may involve improving algorithms, reducing memory
usage, or minimizing I/O operations to achieve better overall performance.

49
Example: If profiling reveals that a program spends a significant amount of time in a sorting
function, developers can optimize the sorting algorithm or use parallel processing to speed up
execution.

B. Key Metrics in Program Performance

To evaluate program performance, developers focus on specific metrics that represent the
efficiency and effectiveness of the program’s use of system resources.
1. Execution Time
• Definition: The total time taken for a program to complete its execution, from the start
to the end of the process. It includes all computation time, I/O operations, and system
calls.
• Relevance: Execution time is one of the most critical performance metrics, as it directly
affects the responsiveness and efficiency of the program. Long execution times may
indicate inefficiencies in algorithm design, excessive I/O, or poor concurrency
handling.
• Optimization Focus: Optimizing execution time may involve improving algorithmic
efficiency, reducing unnecessary computations, and minimizing the time spent on I/O
operations.
2. Memory Footprint
• Definition: The total amount of memory used by a program during its execution,
including memory used for variables, data structures, and dynamically allocated
memory.
• Relevance: A high memory footprint can lead to excessive resource consumption,
memory leaks, or system crashes in memory-constrained environments. Reducing
memory usage is crucial for improving overall system performance, particularly in
embedded or mobile applications.
• Optimization Focus: Memory optimization techniques include reducing the number
of memory allocations, improving memory reuse, using memory pools, and identifying
memory leaks through memory profiling tools.
3. I/O Performance
• Definition: The efficiency with which the program performs input/output operations,
such as reading from or writing to files, databases, or network resources.
• Relevance: Poor I/O performance can severely impact program speed, especially in
data-intensive applications like databases or file servers. Efficient handling of I/O
operations is necessary to ensure the program runs optimally.
• Optimization Focus: Improving I/O performance may involve minimizing disk access,
reducing the number of system calls, optimizing file access patterns, and using
asynchronous I/O techniques to prevent blocking operations.
50
4. Concurrency Performance
• Definition: The ability of a program to efficiently handle multiple threads or processes
running in parallel. Concurrency performance measures how well the program utilizes
multi-core processors and manages parallel tasks.
• Relevance: As systems become increasingly multi-core, efficient handling of
concurrency is essential to improve performance. Poor concurrency performance may
result in thread contention, race conditions, or deadlocks, which degrade system
performance.
• Optimization Focus: Optimizing concurrency involves improving thread
management, using appropriate synchronization mechanisms (such as semaphores,
mutexes, or atomic operations), and ensuring that parallelism is leveraged effectively
without causing overhead.

C. Optimization Techniques
After profiling the program and evaluating its performance, developers can apply a variety of
optimization techniques to improve the program’s resource usage and execution efficiency.
1. Code Optimization
• Definition: The process of refactoring and modifying the code to improve its efficiency.
Code optimization focuses on reducing the execution time, memory usage, and
unnecessary operations within the program.
• Methods:
o Algorithm Refinement: Replacing inefficient algorithms with more efficient
alternatives (e.g., using binary search instead of linear search).
o Loop Optimization: Reducing loop overhead, unrolling loops, or eliminating
nested loops when possible.
o Inlining Functions: Inlining frequently called functions to eliminate the
overhead of function calls.
o Eliminating Redundancies: Identifying and removing redundant calculations
or unnecessary function calls.
2. Algorithm Improvement
• Definition: Improving the performance of a program by choosing more efficient
algorithms. Algorithmic complexity plays a significant role in program performance,
and selecting the right algorithm can dramatically reduce execution time.
• Methods:
o Big-O Analysis: Analyzing the time and space complexity of algorithms to
ensure scalability and efficiency.

51
o Efficient Data Structures: Using appropriate data structures (e.g., hash tables,
trees, graphs) to speed up operations such as searching and sorting.
o Optimized Sorting and Searching: Using efficient algorithms like QuickSort,
MergeSort, or binary search instead of less efficient ones like BubbleSort or
Linear Search.
3. Resource Allocation
• Definition: Efficiently managing system resources such as CPU, memory, and disk to
ensure that the program runs optimally.
• Methods:
o Dynamic Resource Allocation: Allocating memory or CPU resources
dynamically based on real-time demand, using techniques like memory pooling
or thread pooling.
o Load Balancing: Distributing workload evenly across multiple processors or
nodes in a distributed system to prevent overloading a single resource.
o Priority Scheduling: Using priority-based scheduling in multi-threaded or
multi-process environments to allocate CPU time to the most critical tasks.
4. Caching
• Definition: Storing frequently accessed data in faster storage (e.g., RAM) to reduce the
time spent accessing slower storage media, such as disks or network resources.
• Methods:
o In-Memory Caching: Storing computed results or frequently accessed data in
memory to avoid repeated calculations or data retrievals.
o Cache Eviction Policies: Implementing policies like Least Recently Used
(LRU) to manage the cache size and keep it efficient.
o Database Query Caching: Using techniques like query result caching or
connection pooling to optimize database access times.
Example: For a web application, caching user sessions or commonly requested data in memory
(e.g., using Redis or Memcached) can significantly reduce database query time and improve
user experience by providing faster responses.

Conclusion
Evaluating program performance is a critical aspect of software development that involves
identifying inefficiencies, understanding resource usage, and optimizing code to ensure the
program meets its performance objectives. By using profiling tools, analyzing key metrics like
execution time, memory usage, I/O performance, and concurrency efficiency, developers can
gain valuable insights into how well their programs perform. Through various optimization
techniques such as code optimization, algorithm improvement, resource allocation, and
52
caching, developers can enhance program efficiency, reduce resource consumption, and
ultimately deliver better, faster software.

Summary
Computer System Performance Evaluation is a multi-faceted discipline focusing on measuring,
analyzing, and enhancing computer system efficiency. Techniques range from statistical
sampling and workload characterization to advanced modeling and benchmarking.
Key points to remember:
• Performance modeling provides predictions and aids in system design and
optimization.
• Measurement techniques capture data on system operations and are essential for
effective evaluation.
• Statistical techniques help interpret performance data and support informed decision-
making.
• Benchmarking is a practical approach for comparing system performance under
controlled workloads.
• Workload characterization informs tuning and helps predict system behavior under
different demands.
• Performance evaluation in selection and design ensures systems meet performance
requirements.
• Program performance analysis aids developers in identifying bottlenecks and
optimizing code.
Understanding and applying these techniques are essential for designing, selecting, and
maintaining high-performance computer systems.

Questions and Answers

Question 1
Define program profiling and explain its significance in improving the performance of
EU’s student management system.
Answer:
Program profiling is the process of analyzing a program during execution to gather data on
its behavior, such as time spent in various functions, memory usage, and I/O operations.
• Significance for EU’s Student Management System: Profiling can identify
performance bottlenecks in modules like course registration or fee payment. For
instance, if the registration system experiences delays, profiling could reveal inefficient
53
database queries or redundant computations. Optimizing these areas would ensure
faster response times and improve user satisfaction for students and administrators.

Question 2
Describe three key performance metrics and explain their relevance to EU’s library
management system.
Answer:
1. Execution Time: Measures the time taken to complete tasks like searching for a book
or processing a borrowing transaction. Faster execution ensures that students
experience minimal delays.
2. Memory Footprint: Tracks the memory usage of the library system, especially during
peak periods. Efficient memory usage prevents crashes or slowdowns when multiple
users search or borrow books simultaneously.
3. I/O Performance: Assesses the efficiency of file or database access. For example,
ensuring quick access to e-books or database records of borrowed items improves the
overall usability of the library system.

Question 3
Using an example from EU’s online learning platform, explain the role of scalability
testing in program performance evaluation.
Answer:
Scalability testing ensures that a system can handle increased workloads without performance
degradation.
• Example: During online exams, EU’s learning platform might need to handle 10,000
students simultaneously submitting answers. Scalability testing would simulate this
scenario, identifying whether the system can manage the load efficiently. It may reveal
the need for load balancing or the addition of more servers to maintain performance
during such high-demand periods.

Question 4
EU’s finance department wants to optimize the fee payment system. List and explain two
optimization techniques that could improve its performance.
Answer:
1. Caching: Frequently accessed data, such as student fee statuses or bank transaction
confirmations, can be stored temporarily in memory. This reduces the need to
repeatedly query the database, speeding up the system for all users.

54
2. Algorithm Improvement: Refactor the algorithms used for processing payments. For
example, replacing a less efficient batch-processing method with a real-time processing
system could reduce delays and improve transaction success rates.

Question 5
If EU's student portal currently processes fee payments in 10 seconds per transaction and
optimization reduces this time by 40%, how long will it take to process 1,000 payments
after optimization?
Answer:
• Current Time Per Transaction: 10.
• Time Reduction: 10×0.40 = 4 seconds
• Optimized Time Per Transaction: 10 – 4 = 6 seconds
• Total Time for 1,000 Payments:
1,000 × 6 = 6,000 seconds = 100 minutes
Answer:
It will take 100 minutes to process 1,000 payments after optimization.

Question 6
A program currently sorts a dataset of 10,000 elements using Bubble Sort, which has a time
complexity of O(n2). The execution time is 25 seconds. The program is refactored to use Merge
Sort, which has a time complexity of O(n log n).
Estimate the new execution time assuming the sorting operations dominate and other factors
remain constant.

55
Question 7
A program processes 1,000,000 records, each consuming 64 bytes. During optimization:
• Developers reduce memory usage by compressing each record by 50%.
• An additional 2 MB is used for metadata.
Calculate the total memory footprint before and after optimization.
Solution:
1. Initial Memory Usage:
Memory = 1,000,000 × 64 bytes = 64,000,000 bytes = 64 MB.
2. Optimized Memory Usage:
Memory per record = 64 × 0.5 = 32 bytes.
Memory for records=1,000,000 × 32 = 32,000,000 bytes = 32 MB
Total Memory = 32 MB + 2 MB = 34 MB
3. Memory Saved:
Saved Memory = 64 MB − 34 MB = 30 MB
Answer:
The optimized program uses 34 MB, saving 30 MB of memory.

56