Module name: Apply python programming

fundamentals& BDCPC301

RQF Level : 5
Credits: 10
Sector: ICT and MULTIMEDIA
Trade: NETWORKING AND INTERNET
TECHNOLOGY
Module Type: GENERAL

Prepared by Trainer Phocas BIRASHOBOKA

Email: [email protected]
LO 1: PREPARE PYTHON ENVIRONMENT
1.1. Selection of Python tools

Python Programming Overview

1. Definition
Python is a high-level, interpreted programming language known for its simplicity
and readability. Developed by Guido van Rossum and released in 1991, Python
supports multiple programming paradigms, including procedural, object-oriented,
and functional programming.
2. Benefits
• Easy to Learn and Use: Python’s syntax is clear and concise, making it
beginner-friendly.
• Wide Range of Libraries: Python has an extensive standard library and
many third-party libraries for different use cases such as web development,
data science, automation, and more.
• Cross-Platform Compatibility: Python runs on many platforms like
Windows, macOS, and Linux.
• Large Community: Python has a strong, active community, providing
extensive documentation, tutorials, and support.
• Integration Capabilities: Python can be easily integrated with other
languages such as C, C++, and Java, as well as with frameworks like
TensorFlow and Django.
3. Characteristics of Python
• Interpreted: Python code is executed line by line, which makes it easier to
test and debug.
• Dynamically Typed: You don’t need to declare the type of a variable
explicitly; Python determines it at runtime.
• Object-Oriented: Python supports object-oriented programming, allowing
you to define classes and objects to model real-world entities.
• Extensible and Embeddable: Python can be extended with libraries written
in other languages and embedded in applications to provide scripting
capabilities.
• Readable Code: Python emphasizes readability, with a clean syntax that
uses indentation instead of braces or keywords, making the code easier to
maintain.
• Comprehensive Standard Library: Python’s standard library includes
modules for handling file I/O, interacting with web protocols, regular
expressions, and more.
Applications of Python
1. Data Science
Python is widely used in data science due to its simplicity and robust libraries that
make data analysis, manipulation, and visualization easy. Popular libraries for data
science include:
• Pandas: For handling structured data and performing complex operations on
datasets.
• NumPy: Used for numerical computations and working with large, multi-
dimensional arrays.
• Matplotlib & Seaborn: For data visualization, generating plots, charts, and
graphs.
• SciPy: Used for scientific computations like optimization, integration, and
signal processing.
• TensorFlow & PyTorch: Machine learning libraries for building and
deploying predictive models.
2. Software Development
Python is a versatile language used for developing various types of software
applications, ranging from desktop to web-based solutions.
• Web Development: Python frameworks like Django and Flask allow
developers to create robust web applications with minimal effort. Django
offers an all-in-one solution for creating web applications, while Flask is a
lightweight, flexible framework for smaller apps.
• Game Development: Libraries like Pygame make Python suitable for
developing 2D games.
• GUI Applications: Python is used to build graphical user interfaces (GUIs)
with libraries such as Tkinter, PyQt, and Kivy.
• API Development: Python is often used to build RESTful APIs, using
frameworks like FastAPI and Flask.
3. Automation
Python is a powerful tool for automating repetitive tasks, making it highly useful for
system administration, task scheduling, and workflow automation.
• Selenium: Automates web browser interaction for tasks like web scraping or
testing web applications.
• PyAutoGUI: Used to automate keyboard and mouse actions, making it
possible to simulate human interaction with a system.
• Paramiko & Fabric: Tools for automating SSH connections and server
management, often used for remote system administration.
• Task Automation: Python can be used to automate file handling, emails,
and other routine tasks with scripts.
4. Data Analytics
Python is a leading tool in data analytics, offering powerful libraries to process,
analyze, and derive insights from large datasets.
• Pandas: Used to clean, process, and manipulate structured data.
• NumPy: For performing complex mathematical and statistical operations on
numerical data.
• SciPy: Provides algorithms for optimization, integration, and statistics that
are essential for in-depth data analysis.
• Jupyter Notebooks: A web-based tool that allows data analysts to write and
execute code interactively, often used for data exploration and reporting.
Identification of Python Tools
Python offers a wide range of tools and libraries suited for various applications,
from software development to data science. Here are some important Python tools
categorized by their use:
1. Development Environments (IDEs)
• PyCharm: An IDE for Python with advanced code editing features,
debugging, and project management.
• VS Code: A lightweight, versatile editor with Python extensions that include
IntelliSense and debugging.
• Jupyter Notebook: A web-based interactive environment mainly used for
data science and machine learning, allowing users to combine code, text,
and visualizations.
• Spyder: An IDE tailored for data science, including integration with libraries
like NumPy, Pandas, and Matplotlib.
2. Libraries and Frameworks
• Flask & Django: Web development frameworks. Flask is lightweight, while
Django is feature-rich for larger applications.
• NumPy: Used for handling large multi-dimensional arrays and performing
mathematical computations.
• Pandas: Provides high-level data structures for manipulating and analyzing
data.
• Matplotlib & Seaborn: Visualization libraries for generating plots, charts,
and graphs.
• TensorFlow & PyTorch: Deep learning libraries for building machine
learning models.
• Selenium: A browser automation tool for web scraping and automated web
testing.
3. Automation Tools
• PyAutoGUI: Automates mouse and keyboard operations.
• Paramiko: Automates SSH connections for server management.
• Fabric: Used for remote command execution and deployment automation.

1.2. Installation of Python tools

Identification of Computer System Requirements

1. Hardware Requirements
• Processor: Any modern CPU, preferably with multi-core architecture (Intel
Core i3/i5/i7 or AMD Ryzen).
• RAM: Minimum 4 GB, though 8 GB or more is recommended for handling
large datasets in data science or running multiple applications.
• Storage: At least 5 GB of free disk space for Python installation and
libraries. More space will be needed for large projects and datasets.
• Graphics: For tasks like deep learning and scientific computing, a GPU
(NVIDIA with CUDA support) may be required.
2. Software Requirements
• Operating System: Python supports Windows, macOS, and Linux.
• Python Interpreter: Download the latest version of Python (3.x) from the
official Python website.
• Package Manager: Ensure that pip (Python’s package manager) is installed
for managing libraries and dependencies.
• Virtual Environment Tools: Install tools like virtualenv or conda to create
isolated environments.
Install Python Software Tools
1. Install Python
o Download the latest Python version from python.org.
o Run the installer and ensure the “Add Python to PATH” option is
checked.
o Verify installation by running python --version in the command
prompt or terminal.
2. Install Required Libraries
Use pip to install libraries and tools. For example:
bash
Copy code
pip install numpy pandas flask django
3. Install an IDE
o Download and install your preferred IDE (e.g., PyCharm, VS Code).
o Configure it to use the correct Python interpreter.

Configure Python Virtual Environment

1. Install virtualenv
You can install virtualenv to manage isolated environments:
Code
pip install virtualenv
2. Create a Virtual Environment
To create a virtual environment in a project directory:
Code
virtualenv venv
3. Activate the Virtual Environment
o On Windows:
Code
venv\Scripts\activate
o On macOS/Linux:
Code
Source venv/bin/activate
4. Install Packages in the Virtual Environment
Once activated, use pip to install libraries:
Code
pip install flask numpy
5. Deactivate the Virtual Environment
When done, deactivate the environment:
Code
Deactivate
1.3. Testing Python Installation

1. Run Python Version Command After installing Python, verify the

installation by checking the version:

Code:
python --version

Or, if Python is installed as python3 (on macOS or Linux):

Code:
python3 --version

The output should display the installed version, e.g., Python 3.x.x.

2. Check Python Interpreter To confirm the Python interpreter is

working, start the Python interactive shell:

Code:

python

Or, for systems where Python 3 is installed as python3:

Code:
python3

If installed correctly, the Python shell will open, showing something

like:

Code:
Python 3.x.x (default, Mar 1 2024, 10:24:25)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

You can now type basic Python commands in the interactive prompt.

To exit the interpreter, type:

Code:
exit()

3. Test Package Manager (pip) Test if Python’s package manager, pip, is

installed and working:

Code:
pip --version
The output should display the version of pip and the Python directory
it's associated with, e.g., pip 23.0.1 from /path/to/python.

To further verify that pip works, try installing a small package like
requests:

Code:
pip install requests

After the installation, you can check if the package was installed
correctly:

Code:
python -m pip show requests

This ensures that Python, the interpreter, and the package manager are
properly installed and functioning.

Learning outcome 2: Write basic python program

LO 2: WRITE BASIC PYTHON PROGRAM

2.1. Applying python basic concepts

Data Types

• Definition: Data types represent the type of data that can be stored
and manipulated within a program.
• Examples:
o Integers: Whole numbers (e.g., 5, -10, 42)
o Float: Decimal numbers (e.g., 3.14, -0.5)
o Strings: Textual data (e.g., "Hello, world!", "Python")
o Booleans: True/False values (True, False)
o Lists: Ordered collections of items (e.g., [1, 2, 3], ['a', 'b', 'c'])
o Dictionaries: Key-value pairs (e.g., {'name': 'John', 'age': 30})

2. Variables

• Definition: Variables are used to store data values, acting as

containers for information that can be referenced later.
• Example:

Code
x = 10 # x is a variable storing the integer value 10
name = "Alice" # name is a variable storing the string "Alice"

3. Comments
• Definition: Comments are lines of text that are ignored by the Python
interpreter. They are used to explain code or leave notes for
developers.
• Types:
o Single-line comment: Starts with #.

Code :
# This is a single-line comment

o Multi-line comment: Uses triple quotes (''' or """).

Code :
'''
This is a multi-line comment.
It spans across several lines.
''''''

4. Operators

• Definition: Operators are symbols used to perform operations on

variables and values.
• Types:
o Arithmetic Operators: For basic math operations (+, -, *, /, %,
**).

Code:
result = 10 + 5 # Addition

o Comparison Operators: Compare values (==, !=, >, <, >=, <=).

Code:
is_equal = 10 == 5 # False

o Logical Operators: Combine boolean expressions (and, or, not).

Code:
condition = True and False # False

o Assignment Operators: Assign values to variables (=, +=, -=,

etc.).

Code:
x = 10
x += 5 # x becomes 15

2.2. Applying python control structures

1. Conditional Statements

• Definition: Conditional statements allow the program to make

decisions and execute certain blocks of code based on conditions.
• Examples:

Code:
x = 10
if x > 5:
print("x is greater than 5")
elif x == 5:
print("x is equal to 5")
else:
print("x is less than 5")

• Explanation:
o if: Checks if the condition is True.
o elif: Allows for multiple conditions to be checked.
o else: Executes if none of the above conditions are met.

2. Looping Statements

• Definition: Looping statements allow you to execute a block of code

multiple times.
• Types:
o For Loop: Iterates over a sequence (like a list or string).

Code :
fruits = ["apple", "banana", "cherry"]
for fruit in fruits:
print(fruit)

▪Explanation: Loops through each item in the list fruits

and prints it.
o While Loop: Repeats as long as a condition is True.

Code:
i=1
while i < 5:
print(i)
i += 1 # Increment i

▪ Explanation: Prints numbers from 1 to 4, increasing i

with each iteration.

3. Use of Jump Statements

• Definition: Jump statements control the flow of loops, either skipping

iterations, breaking the loop, or doing nothing.
• Break: Exits the loop immediately when encountered.

Code :
for i in range(1, 10):
if i == 5:
break # Loop stops when i is 5
print(i)
o Explanation: This loop prints numbers from 1 to 4, and stops
when i equals 5.
• Continue: Skips the current iteration and continues with the next
one.

Code:
for i in range(1, 6):
if i == 3:
continue # Skip the number 3
print(i)

o Explanation: Prints numbers from 1 to 5 but skips 3.

• Pass: Does nothing; it acts as a placeholder for future code.

Code:
for i in range(1, 6):
if i == 3:
pass # Placeholder, does nothing
print(i)

o Explanation: Prints numbers from 1 to 5. pass doesn't affect the

flow; it’s used when you want to add logic later.

2.3. Applying functions in Python

1. Definition of Function

• Definition: A function is a block of reusable code that performs a

specific task. It is defined using the def keyword and can accept inputs
(called parameters) and return outputs.

Example:

Code:
def greet(name):
return f"Hello, {name}!"

print(greet("Alice")) # Output: Hello, Alice!

• Explanation: The function greet takes an argument name, and returns

a greeting string.

2. Types of Functions

a) Built-in Functions
• Definition: These are functions that are already provided by Python,
and you can use them directly without defining them.
• Examples:
o print(): Outputs the given argument.
o len(): Returns the length of a sequence.
o type(): Returns the type of a variable.
o max(), min(): Return the maximum and minimum values from a
sequence.

Code:

numbers = [1, 2, 3, 4, 5]
print(len(numbers)) # Output: 5
print(max(numbers)) # Output: 5
print(type(numbers)) # Output: <class 'list'>

b) User-Defined Functions

• Definition: Functions created by the user to perform specific tasks.

• Syntax:

Code:
def function_name(parameters):
# Function body
return result

• Example:

Code:
def add_numbers(a, b):
return a + b

result = add_numbers(3, 4)
print(result) # Output: 7

• Explanation: In this example, add_numbers is a user-defined function

that takes two parameters and returns their sum.

Example: Using Both Built-in and User-Defined Functions

Together
Code:
def find_max(numbers):
return max(numbers) # Using the built-in max() function

nums = [10, 20, 30, 5, 15]

print(find_max(nums)) # Output: 30
• Explanation: Here, the find_max function is a user-defined function
that uses Python’s built-in max() function to find the largest number in
a list.

Create a Function

• A function is created using the def keyword, followed by the function

name, parentheses for parameters, and a block of code to execute.

Code:
def greet(name):
print(f"Hello, {name}!")

Explanation: This greet function takes one argument (name) and prints a
greeting.

2. Arguments

• Arguments are values passed to the function when it is called.

• Example:

Code:

def add(a, b):

return a + b

result = add(3, 5) # 3 and 5 are arguments

print(result) # Output: 8

3. Default Parameter Value

• You can give parameters default values. If the caller doesn’t provide a
value for that parameter, the default will be used.
• Example:

Code:
def greet(name="Guest"):
print(f"Hello, {name}!")

greet("Alice") # Output: Hello, Alice!

greet() # Output: Hello, Guest!

Explanation: Here, if no argument is passed to greet(), the function uses the

default value "Guest".

4. Passing a List as an Argument

• You can pass a list as an argument to a function and perform
operations on it.
• Example:

Code:
def sum_list(numbers):
return sum(numbers)

my_list = [1, 2, 3, 4, 5]
result = sum_list(my_list)
print(result) # Output: 15

Explanation: The list my_list is passed as an argument to sum_list, which

calculates and returns the sum of the list.

5. Calling a Function

• To call a function, simply use its name followed by parentheses, and

pass any required arguments.
• Example:

Code:
def multiply(a, b):
return a * b

result = multiply(3, 4) # Function is called with arguments 3 and 4

print(result) # Output: 12

Complete Example

Here’s a function that incorporates arguments, default parameter values,

and passing a list as an argument:

Code:
def process_list(numbers, multiplier=2):
"""
Multiplies each element of the list by the given multiplier (default is 2).
"""
return [number * multiplier for number in numbers]

my_list = [1, 2, 3, 4]
print(process_list(my_list)) # Output: [2, 4, 6, 8]
print(process_list(my_list, 3)) # Output: [3, 6, 9, 12]

• Explanation:
o process_list takes a list numbers and multiplies each element by
multiplier.
o If no multiplier is provided, it defaults to 2.
Apply special purpose functions
1. Lambda Functions

• Definition: A lambda function is an anonymous function in Python,

defined using the lambda keyword. It can take any number of
arguments but only contains a single expression.
• Syntax:

Code:
lambda arguments: expression

• Example:

Code:
add = lambda x, y: x + y
print(add(5, 3)) # Output: 8

Explanation: Here, add is a lambda function that takes two arguments (x, y)
and returns their sum.

2. Python Generators

• Definition: Generators are functions that return an iterator and allow

you to iterate through a sequence of values without storing them all in
memory. They are defined like normal functions but use yield instead
of return.
• Example:

Code:
def count_up_to(max):
count = 1
while count <= max:
yield count # Return the value, but keep the function state
count += 1

for number in count_up_to(5):

print(number)

Explanation: The generator count_up_to yields numbers from 1 to 5. Unlike

lists, generators are lazy, meaning they produce items one at a time and
only when requested.

3. Python Closures

• Definition: A closure is a function that remembers the values from its

enclosing scope, even if the enclosing function has finished execution.
• Example:

Code:

def outer_function(msg):
def inner_function():
print(msg) # `msg` is remembered by `inner_function`
return inner_function

hello_func = outer_function("Hello, World!")

hello_func() # Output: Hello, World!

Explanation: The inner function remembers the value of msg from the outer
function, even after outer_function has finished.

4. Python Decorators

• Definition: A decorator is a function that wraps another function to

extend or alter its behavior without modifying the original function’s
code.
• Example:

Code:
def my_decorator(func):
def wrapper():
print("Something before the function is called.")
func()
print("Something after the function is called.")
return wrapper
@my_decorator # Using the decorator
def say_hello():
print("Hello!")
say_hello()

Output:

Code:

Something before the function is called.

Hello!
Something after the function is called.

Explanation: The @my_decorator syntax modifies the behavior of the say_hello

function by wrapping it in another function (wrapper).

5. Recursive Function

• Definition: A recursive function is one that calls itself to solve smaller

instances of the same problem.
• Ex
• 0ample (Factorial calculation):

Code:
def factorial(n):
if n == 1:
return 1
else:
return n * factorial(n - 1)

print(factorial(5)) # Output: 120

Explanation: The factorial function calls itself with a smaller value until it
reaches the base case (n == 1), where it returns 1.

6. Higher-Order Functions

• Definition: A higher-order function is a function that either takes

another function as an argument, returns a function, or both.
• Example:

Code:
def apply_function(func, value):
return func(value)

def square(x):
return x * x

print(apply_function(square, 4)) # Output: 16

Explanation: The apply_function takes another function (square) as an

argument and applies it to the given value (4).

Complete Example

Let’s combine multiple concepts: a higher-order function, lambda function,

and a recursive function.

Code:
# Higher-order function using a lambda and recursion
def apply_operation(operation, n):
if n == 0:
return 0
else:
return operation(n) + apply_operation(operation, n - 1)

# Using a lambda function

result = apply_operation(lambda x: x * x, 5) # Sum of squares of 1 to 5
print(result) # Output: 55 (1^2 + 2^2 + 3^2 + 4^2 + 5^2)
Summary

• Lambda functions provide quick anonymous functions for simple

operations.
• Generators allow for memory-efficient iterating.
• Closures remember the context in which they were created.
• Decorators allow for modifying the behavior of functions.
• Recursive functions solve problems by breaking them down into
smaller sub-problems.
• Higher-order functions can take or return other functions, enabling
flexible and reusable code

2.4. Applying Python Collections

Applying Python Collections

1. Lists

• Definition: Lists are ordered, mutable (changeable) collections of

items, allowing duplicate values.
• Example:

Code:

fruits = ["apple", "banana", "cherry"]

print(fruits) # Output: ['apple', 'banana', 'cherry']

fruits.append("orange") # Adding an item

print(fruits) # Output: ['apple', 'banana', 'cherry', 'orange']

Key Features:

• Ordered
• Mutable
• Allows duplicate values

2. Tuples

• Definition: Tuples are ordered, immutable collections of items. Once

created, the items cannot be modified.
• Example:

Code:
coordinates = (10, 20)

print(coordinates) # Output: (10, 20)

# coordinates[0] = 30 # This will raise an error, as tuples are immutable

Key Features:

• Ordered
• Immutable
• Allows duplicate values

3. Dictionaries

• Definition: Dictionaries are unordered collections of key-value pairs.

They are mutable and do not allow duplicate keys.
• Example:

Code:

person = {"name": "Alice", "age": 30}

print(person["name"]) # Output: Alice

person["age"] = 31 # Modifying value

print(person) # Output: {'name': 'Alice', 'age': 31}

Key Features:

• Unordered (as of Python 3.7, dictionaries maintain insertion order)

• Mutable
• Unique keys

4. Sets

• Definition: Sets are unordered collections of unique items, meaning

no duplicates are allowed.
• Example:

Code:

numbers = {1, 2, 3, 3} # Duplicate 3 will be removed

print(numbers) # Output: {1, 2, 3}

numbers.add(4) # Adding an item

print(numbers) # Output: {1, 2, 3, 4}

Key Features:

• Unordered
• Mutable
• Unique elements only

5. Frozen Set

• Definition: Frozen sets are similar to sets but are immutable,

meaning their elements cannot be changed once assigned.
• Example:

Code:

frozen_numbers = frozenset([1, 2, 3, 3])

print(frozen_numbers) # Output: frozenset({1, 2, 3})

# frozen_numbers.add(4) # This will raise an error, as frozen sets are

immutable

Key Features:

• Unordered
• Immutable
• Unique elements only

6. ChainMaps

• Definition: ChainMap groups multiple dictionaries into a single view

for lookup purposes. It does not merge them but provides a single
interface to access the keys and values.
• Example:

Code:

from collections import ChainMap

dict1 = {"a": 1, "b": 2}

dict2 = {"b": 3, "c": 4}

chain = ChainMap(dict1, dict2)

print(chain["b"]) # Output: 2 (takes from dict1, as it appears first)

print(chain["c"]) # Output: 4 (found in dict2)

Key Features:

• Groups multiple dictionaries

• Provides combined access without merging them

7. Deques

• Definition: Deques (double-ended queues) are similar to lists but

allow adding and removing items from both ends efficiently.
• Example:

Code:

from collections import deque

dq = deque([1, 2, 3])

dq.append(4) # Add to the right end

dq.appendleft(0) # Add to the left end

print(dq) # Output: deque([0, 1, 2, 3, 4])

dq.pop() # Remove from the right end

dq.popleft() # Remove from the left end

print(dq) # Output: deque([1, 2, 3])

Key Features:

• Mutable
• Fast append and pop operations from both ends
• Used in situations where queue or stack operations are needed

Summary of Collection Types:

Collection Duplicate Unique
Order Mutability
Type Elements Elements
List Ordered Mutable Allowed Not required
Tuple Ordered Immutable Allowed Not required
Keys must be
Dictionary Unordered Mutable Not required
unique
Set Unordered Mutable Not allowed Required
Frozen Set Unordered Immutable Not allowed Required
ChainMap N/A Mutable Allowed Not required
Deque Ordered Mutable Allowed Not required

These collections allow for a wide range of data storage and manipulation
techniques, depending on your needs (e.g., efficiency in adding/removing,
uniqueness, immutability).

Specialized tools from the collections module

The collections module in Python provides specialized tools to handle

common programming tasks more efficiently. Let’s explore three key tools:
Counter, OrderedDict, and defaultdict.

1. Counter

• Definition: A Counter is a dictionary subclass that counts the

frequency of elements in a collection (like lists, strings, or tuples).
• Example:

Code:

from collections import Counter

fruits = ["apple", "banana", "apple", "orange", "banana", "apple"]

fruit_counter = Counter(fruits)

print(fruit_counter) # Output: Counter({'apple': 3, 'banana': 2, 'orange': 1})

# Access the count of a specific item

print(fruit_counter["apple"]) # Output: 3

print(fruit_counter["cherry"]) # Output: 0 (even if it's not in the list)

Key Features:

• Automatically counts elements.

• Provides methods like most_common() to get the most frequent items.
• Can handle missing keys (returns 0 if a key is not found).

2. OrderedDict

• Definition: An OrderedDict is a dictionary that remembers the order

in which keys are inserted. Starting from Python 3.7, regular
dictionaries also maintain insertion order, but OrderedDict has
additional methods for reordering elements.
• Example:

Code:

from collections import OrderedDict

ordered_dict = OrderedDict()

ordered_dict["apple"] = 1

ordered_dict["banana"] = 2

ordered_dict["cherry"] = 3

print(ordered_dict) # Output: OrderedDict([('apple', 1), ('banana', 2),

('cherry', 3)])

# Rearranging the order

ordered_dict.move_to_end("apple") # Moves 'apple' to the end

print(ordered_dict) # Output: OrderedDict([('banana', 2), ('cherry', 3),

('apple', 1)])

Key Features:

• Maintains the insertion order of keys.

• Provides methods like move_to_end() and popitem(last=False) to
manipulate order.
• Useful when you need order-specific operations or algorithms.

3. defaultdict
• Definition: A defaultdict is a dictionary subclass that provides a
default value for a key if the key hasn’t been set yet. This eliminates
the need to check if a key exists before adding a value.
• Example:

Code:

from collections import defaultdict

# Initialize defaultdict with int, which defaults missing values to 0

word_count = defaultdict(int)

sentence = "hello world hello everyone"

words = sentence.split()

for word in words:

word_count[word] += 1

print(word_count) # Output: defaultdict(<class 'int'>, {'hello': 2, 'world': 1,

'everyone': 1})

Key Features:

• Provides a default value for missing keys (based on the argument

passed during initialization).
• Automatically creates missing dictionary entries on the fly.
• Can use any callable as a default factory (like int, list, or a custom
function).
• Another Example (list as the default factory):

Code:

neighbors = defaultdict(list)

neighbors["A"].append("B")

neighbors["A"].append("C")

neighbors["B"].append("A")

print(neighbors)

# Output: defaultdict(<class 'list'>, {'A': ['B', 'C'], 'B': ['A']})

Explanation: In this example, if a key doesn’t exist, defaultdict
automatically creates an empty list and allows you to append values without
checking if the key exists first.

Summary of collections Specialized Tools:

Tool Description Common Use

Counts the frequency of Counting occurrences of items in
Counter
elements in a collection. a list or characters in a string.
Maintains the order in When the order of insertion needs
OrderedDict
which items are inserted. to be preserved or manipulated.
Provides default values for Eliminating the need to check for
defaultdict
missing dictionary keys. key existence before accessing.

These tools from the collections module make it easier to work with complex
data structures and can often result in more concise and readable code.

Perform common operations

Perform common operations

Let's go over how to perform common operations such as adding/removing

elements, accessing/iterating, filtering/sorting, set operations, counting,
and stack/queue operations in Python using built-in data structures and
tools.

1. Adding and Removing Elements

Applicable to Lists, Sets, Dictionaries, Deques.

• Lists:

Code:
fruits = ["apple", "banana"]
fruits.append("orange") # Adding an element
print(fruits) # Output: ['apple', 'banana', 'orange']

fruits.remove("banana") # Removing an element

print(fruits) # Output: ['apple', 'orange']

• Sets:

Code:
fruits_set = {"apple", "banana"}
fruits_set.add("orange") # Adding an element
print(fruits_set) # Output: {'apple', 'orange', 'banana'}
fruits_set.discard("banana") # Removing an element (no error if not found)
print(fruits_set) # Output: {'apple', 'orange'}

• Dictionaries:

Code:
person = {"name": "Alice", "age": 30}
person["city"] = "New York" # Adding an element
print(person) # Output: {'name': 'Alice', 'age': 30, 'city': 'New York'}

person.pop("age") # Removing an element

print(person) # Output: {'name': 'Alice', 'city': 'New York'}

• Deques:

Code:
from collections import deque
dq = deque([1, 2, 3])
dq.append(4) # Add to the right
dq.appendleft(0) # Add to the left
print(dq) # Output: deque([0, 1, 2, 3, 4])

dq.pop() # Remove from the right

dq.popleft() # Remove from the left
print(dq) # Output: deque([1, 2, 3])

2. Accessing and Iterating Over Elements

Applicable to Lists, Tuples, Dictionaries, Sets.

• Lists and Tuples:

Code:

numbers = [10, 20, 30, 40]

print(numbers[0]) # Accessing the first element, Output: 10

for num in numbers:

print(num) # Iterating over the list, prints: 10, 20, 30, 40

• Dictionaries:

Code:
person = {"name": "Alice", "age": 30}
print(person["name"]) # Accessing a value, Output: Alice

for key, value in person.items():

print(f"{key}: {value}") # Iterating over dictionary

• Sets (unordered, no indexing):

Code:
fruits_set = {"apple", "banana", "orange"}
for fruit in fruits_set:
print(fruit) # Iterating over the set

3. Filtering and Sorting

Applicable to Lists, Sets, Dictionaries.

• Filtering (List Comprehension):

Code:
numbers = [1, 2, 3, 4, 5]
even_numbers = [num for num in numbers if num % 2 == 0]
print(even_numbers) # Output: [2, 4]

• Sorting (Lists and Sets):

Code:
fruits = ["banana", "apple", "orange"]
fruits_sorted = sorted(fruits)
print(fruits_sorted) # Output: ['apple', 'banana', 'orange']
numbers = {3, 1, 2}
sorted_numbers = sorted(numbers)
print(sorted_numbers) # Output: [1, 2, 3]

• Sorting by Dictionary Keys or Values:

Code:
person = {"name": "Alice", "age": 30, "city": "New York"}
sorted_by_key = sorted(person) # Sorts by keys
print(sorted_by_key) # Output: ['age', 'city', 'name']

4. Set Operations and Counting

Applicable to Sets, Counter from collections module.

• Set Operations (Union, Intersection, Difference):

Code:
set1 = {1, 2, 3}
set2 = {3, 4, 5}
print(set1.union(set2)) # Output: {1, 2, 3, 4, 5}
print(set1.intersection(set2)) # Output: {3}
print(set1.difference(set2)) # Output: {1, 2}

• Counting Elements (Using Counter):

Code:

from collections import Counter

words = ["apple", "banana", "apple", "orange", "banana", "apple"]

count = Counter(words)
print(count) # Output: Counter({'apple': 3, 'banana': 2, 'orange': 1})

5. Stack and Queue Operations

Stack (Last In First Out - LIFO) and Queue (First In First Out - FIFO)
operations are typically done using lists, deques, or dedicated modules like
queue.

• Stack (Using List or Deque):

Code:
stack = [1, 2, 3]
stack.append(4) # Push to the stack
print(stack) # Output: [1, 2, 3, 4]
stack.pop() # Pop from the stack
print(stack) # Output: [1, 2, 3]

• Queue (Using List or Deque):

Code:
from collections import deque
queue = deque([1, 2, 3])
queue.append(4) # Enqueue (add to the end)
print(queue) # Output: deque([1, 2, 3, 4])

queue.popleft() # Dequeue (remove from the front)

print(queue) # Output: deque([2, 3, 4])

• Priority Queue (Using heapq):

Code:
import heapq
pq = []
heapq.heappush(pq, (1, "apple")) # (priority, item)
heapq.heappush(pq, (3, "banana"))
heapq.heappush(pq, (2, "orange"))
print(heapq.heappop(pq)) # Output: (1, 'apple') -> smallest priority element

Summary of Operations:

• Adding/Removing: Lists (append/remove), Sets (add/discard),

Dictionaries (add/remove pairs), Deques (append/pop).
• Access/Iterate: Lists and Tuples (indexing), Dictionaries
(keys/values), Sets (iterate).
• Filtering/Sorting: Lists and Sets can be filtered and sorted easily
using comprehensions and the sorted() function.
• Set Operations: Union, Intersection, Difference for mathematical set-
like operations.
• Stack/Queue: Use Lists/Deques for stack (LIFO) and queue (FIFO)
operations; heapq for priority queues.
2.5. File handling
Python File Handling and Key Libraries:

Python provides several libraries and tools to work with files and directories.
Key libraries for file handling include os, pathlib, shutil, and pandas.

1. Basic File Handling in Python

File handling in Python allows you to open, read, write, and close files. The
basic file operations can be done using Python’s built-in open() function.

File Operations

• Opening a file:

Code:

file = open('example.txt', 'r') # 'r' for reading, 'w' for writing, 'a' for appending

• Reading a file:

Code:

content = file.read() # Reads the entire file

print(content)

file.close() # Always close the file after operations

• Writing to a file:

Code:

file = open('example.txt', 'w')

file.write("Hello, World!") # Writes the string to the file

file.close()

• Using with for file handling (automatically closes the file):

Code:

with open('example.txt', 'r') as file:

content = file.read()
print(content)

2. os Library

The os module provides a way to interact with the operating system,

allowing you to work with directories, files, and system commands.

Key Functions:

• Check if a file or directory exists:

Code:

import os

print(os.path.exists('example.txt')) # Output: True if the file exists

• Creating and removing directories:

Code:

os.mkdir('new_directory') # Create a new directory

os.rmdir('new_directory') # Remove a directory

• Listing files in a directory:

Code:

files = os.listdir() # Lists files in the current directory

print(files)

• Getting file information:

Code:

file_info = os.stat('example.txt') # Get file metadata

print(file_info.st_size) # Output: size of the file in bytes

3. pathlib Library

pathlib is an object-oriented approach to handling filesystem paths. It

makes file and directory manipulation more intuitive and readable.
Key Functions:

• Creating a path:

Code:

from pathlib import Path

path = Path('example.txt')

print(path.exists()) # Check if file exists

• Reading and writing files:

Code:

# Writing to a file

path.write_text("Hello, World!")

# Reading from a file

content = path.read_text()

print(content)

• Working with directories:

Code:

directory = Path('new_directory')

directory.mkdir() # Create a directory

• Traversing directories:

Code:

for file in Path('.').iterdir(): # Iterates through the current directory

print(file)

4. shutil Library

shutil is used for high-level file and directory operations such as copying,
moving, and deleting files or directories.
Key Functions:

• Copying files:

Code:

import shutil

shutil.copy('example.txt', 'copy_of_example.txt') # Copy a file

• Moving files:

Code:

shutil.move('example.txt', 'moved_example.txt') # Move (or rename) a file

• Deleting directories:

Code:

shutil.rmtree('directory_to_remove') # Recursively delete a directory and its

contents

5. pandas Library

While pandas is primarily a data manipulation library, it offers powerful file-

handling capabilities, especially for CSV, Excel, and other tabular formats.

Key Functions:

• Reading from CSV files:

Code:

import pandas as pd

df = pd.read_csv('data.csv') # Read CSV into a DataFrame

print(df)

• Writing to CSV files:

Code:

df.to_csv('output.csv', index=False) # Write DataFrame to a CSV file without

index
• Reading and Writing Excel files:

Code:

df = pd.read_excel('data.xlsx') # Read Excel file into a DataFrame

df.to_excel('output.xlsx') # Write DataFrame to an Excel file

Working with Large Datasets:

pandas can handle very large datasets efficiently, enabling chunked reads
for large files that don’t fit in memory:

code:

chunk_size = 1000

for chunk in pd.read_csv('large_data.csv', chunksize=chunk_size):

print(chunk.head()) # Process data in chunks

Summary of File Handling Libraries:

Library Purpose Common Use Cases

Creating/removing directories,
os Low-level file system operations
listing files
Object-oriented file and File paths manipulation, creating
pathlib
directory path handling paths
High-level file and directory Copying, moving, and deleting
shutil
operations files and directories
Data manipulation and Reading/writing CSV, Excel,
pandas
reading/writing tabular files handling large datasets

These libraries offer a comprehensive suite of tools for working with files and
directories in Python, each suited to specific tasks, whether you’re dealing
with basic file I/O or complex data manipulation.

Practice to read filed file

Open a File

You can open a file using Python’s built-in open() function. The most
common file modes include:
• r: Read (default mode)
• w: Write (overwrite if the file exists)
• a: Append (add to the end of the file)
• rb, wb: Read/Write in binary mode

Example of Opening and Reading a File:

code

# Open a file in read mode

file = open('example.txt', 'r')

# Read the content of the file

content = file.read()

# Print the content

print(content)

# Close the file to free up resources

file.close()

2. Using with for File Handling:

Using the with statement ensures that the file is automatically closed after
operations, even if an exception occurs.

code

with open('example.txt', 'r') as file:

content = file.read()

print(content) # Output: Content of the file is printed

# File is automatically closed after exiting the 'with' block

3. Check File Permissions

To check the permissions of a file, you can use the os or pathlib library.

Using os module:
The os module provides a function called os.access() to check for file
permissions.

• os.R_OK: Readable
• os.W_OK: Writable
• os.X_OK: Executable

Example:

code

import os

file_path = 'example.txt'

# Check if the file is readable

if os.access(file_path, os.R_OK):

print(f"{file_path} is readable.")

else:

print(f"{file_path} is not readable.")

# Check if the file is writable

if os.access(file_path, os.W_OK):

print(f"{file_path} is writable.")

else:

print(f"{file_path} is not writable.")

Using pathlib module:

pathlib also allows checking file permissions.

code

from pathlib import Path

file_path = Path('example.txt')

# Check if the file is readable

if file_path.is_file() and file_path.exists() and os.access(file_path, os.R_OK):

print(f"{file_path} is readable.")

else:

print(f"{file_path} is not readable.")

4. File Permissions in Unix-like Systems

If you're working on a Unix-like system (Linux, macOS), file permissions are

often represented as a set of rwx (read, write, execute) flags for the owner,
group, and others. You can use os.stat() to check these permission bits.

code

import os

import stat

file_stat = os.stat('example.txt')

# Convert file permissions to octal

permissions = oct(file_stat.st_mode)[-3:]

print(f"File permissions: {permissions}")

Example Output:

• 755 means the owner has read, write, and execute permissions, and
the group and others have read and execute permissions.

Summary of Steps:

1. Open a file using open() or with open().

2. Read the file using .read() or .readlines().
3. Check permissions using os.access() or pathlib.

Perform write/create file

1. Create and Write to Files

Python’s open() function can be used to create a new file if it doesn’t exist or
write to an existing file.

Create a New File and Write to It:

When you open a file in write mode ('w'), Python will:

• Create the file if it does not exist.

• Overwrite the file if it already exists.

python

Copy code

# Open a file in write mode

file = open('new_file.txt', 'w')

# Write content to the file

file.write("This is a new file created with Python.")

# Close the file

file.close()

Alternatively, you can use the with statement to ensure that the file is
properly closed after writing:

python

Copy code

with open('new_file.txt', 'w') as file:

file.write("This is a new file created with the 'with' statement.")

Append to an Existing File:

To append to an existing file without overwriting its content, open the file in
append mode ('a'):

python

Copy code
with open('existing_file.txt', 'a') as file:

file.write("\nAdding a new line to the existing file.")

2. Delete a File

You can delete a file using the os module’s remove() function.

Remove a File:

python

Copy code

import os

file_path = 'new_file.txt'

# Check if file exists before removing it

if os.path.exists(file_path):

os.remove(file_path) # Delete the file

print(f"{file_path} has been deleted.")

else:

print(f"{file_path} does not exist.")

3. Delete a Directory (Folder)

You can delete directories using the os or shutil module. You need to be
careful when deleting directories to ensure you don’t accidentally delete
important data.

Remove an Empty Directory:

The os.rmdir() function is used to remove an empty directory.

python
Copy code

directory = 'empty_folder'

# Check if directory exists before removing it

if os.path.exists(directory):

os.rmdir(directory) # Remove the directory

print(f"{directory} has been deleted.")

else:

print(f"{directory} does not exist.")

Remove a Directory with Contents:

To remove a directory and all its contents (subdirectories and files), use
shutil.rmtree().

python

Copy code

import shutil

directory = 'non_empty_folder'

# Check if the directory exists before removing it

if os.path.exists(directory):

shutil.rmtree(directory) # Recursively delete the directory and its

contents

print(f"{directory} and its contents have been deleted.")

else:

print(f"{directory} does not exist.")

Summary of Steps:

• Create and write to files using open() in 'w' mode.

• Append to existing files using 'a' mode.
• Delete a file using os.remove().
• Remove an empty directory using os.rmdir().
• Remove a directory with contents using shutil.rmtree()

Apply python best practices

1. Readability and Style

Readability is crucial in Python. Follow the PEP 8 style guide for better code
quality.

Key Practices:

• Use descriptive variable and function names:

python

Copy code

def calculate_total_price(items):

total = 0

for item in items:

total += item['price']

return total

• Indentation: Always use 4 spaces per indentation level (avoid using

tabs).
• Line length: Limit all lines to a maximum of 79 characters.
• Spacing:
o Use spaces around operators and after commas, not inside
parentheses:

Code:

total = price * quantity + tax

• Use docstrings and comments:

o Use docstrings to describe functions and modules.

Code:
def fetch_user_data(user_id):

"""

Fetch user data from the database by user ID.

:param user_id: ID of the user

:return: User data dictionary

"""

pass # Implementation here

o Use comments to clarify complex sections of code, but avoid

over-commenting obvious things.

2. Use of Built-in Features

Python provides many built-in functions and standard libraries that are
optimized and tested, so you should prefer them over writing your own
implementations.

Key Practices:

• Use built-in functions:

o Instead of using loops to find the maximum value in a list, use
max():

Code:

numbers = [3, 1, 4, 1, 5, 9]

max_number = max(numbers)

• List comprehensions:
o Use list comprehensions for readability and performance over
for loops:

Code:

# Using for loop

squares = []

for x in range(10):
squares.append(x**2)

# Using list comprehension

squares = [x**2 for x in range(10)]

• Dictionary comprehensions:
o Example of dictionary comprehension:

Code:

prices = {'apple': 3, 'banana': 2, 'cherry': 5}

discounted_prices = {key: value * 0.9 for key, value in prices.items()}

• Use built-in enumerate() for indexing:

Code:

fruits = ['apple', 'banana', 'cherry']

for index, fruit in enumerate(fruits, start=1):

print(f"{index}. {fruit}")

• Use zip() to iterate over two lists simultaneously:

Code:

names = ['Alice', 'Bob', 'Charlie']

scores = [85, 90, 88]

for name, score in zip(names, scores):

print(f"{name} scored {score}")

3. Efficiency and Memory Usage

Efficient code reduces computation time and conserves memory.

Key Practices:
• Use generators for large datasets:
o Generators are memory-efficient as they yield items one at a
time instead of storing the entire sequence in memory.

Code:

def number_generator(n):

for i in range(n):

yield i

gen = number_generator(1000)

print(next(gen)) # Output: 0

print(next(gen)) # Output: 1

• Use join() for string concatenation:

o Instead of concatenating strings in a loop, use ''.join(). It’s faster
and more efficient.

Code:

words = ['Python', 'is', 'great']

sentence = ' '.join(words) # Output: "Python is great"

• Avoid unnecessary object creation:

o For example, use a single mutable list instead of creating new
lists in loops.
• Use set() for membership tests:
o set has average O(1) time complexity for lookups, whereas list
has O(n).

Code:

items = [1, 2, 3, 4, 5]

item_set = set(items)

if 3 in item_set: # Faster than checking in a list

print("Found!")
4. Error Handling and Testing

Good error handling improves the reliability of your code, while testing
ensures its correctness.

Key Practices:

• Use try/except blocks for error handling:

o Catch specific exceptions and handle them gracefully. Avoid
catching general exceptions unless absolutely necessary (except
Exception).

Code:

try:

with open('file.txt', 'r') as file:

content = file.read()

except FileNotFoundError:

print("File not found. Please check the file path.")

except Exception as e:

print(f"An error occurred: {e}")

• Use finally to clean up resources:

Code:

try:

file = open('file.txt', 'r')

content = file.read()

except FileNotFoundError:

print("File not found.")

finally:

file.close() # Ensures the file is closed even if an error occurs

• Testing:
o Write unit tests using the built-in unittest module or pytest.
Testing ensures that changes in your code don’t introduce bugs.

Code:

import unittest

def add(a, b):

return a + b

class TestAddFunction(unittest.TestCase):

def test_add(self):

self.assertEqual(add(2, 3), 5)

self.assertEqual(add(-1, 1), 0)

if __name__ == '__main__':

unittest.main()

• Use assertions for debugging:

o During development, assertions help catch bugs by checking if
certain conditions are met:

Code:

def divide(a, b):

assert b != 0, "Division by zero is not allowed"

return a / b

Summary of Python Best Practices:

1. Readability and Style: Follow PEP 8, use descriptive names, proper

indentation, and comments.
2. Use of Built-in Features: Leverage Python's built-in functions and
comprehensions for better performance.
3. Efficiency and Memory Usage: Use generators, optimize string
operations, and prefer set() for membership testing.
4. Error Handling and Testing: Use specific try/except blocks and write
unit tests to ensure your code works correctly under all conditions.

LO.3: APPLY OBJECT-DRIVEN IN PYTHON

3.1. Applying OOP Concepts

1. Object and Python Class

A class in Python is a blueprint for creating objects. An object is an

instance of a class that contains attributes (data) and methods (functions).

Defining a Class:

python

Copy code

class Car:

# Class constructor (initializer)

def __init__(self, make, model, year):

self.make = make # Attribute (instance variable)

self.model = model

self.year = year

# Method (function inside a class)

def start_engine(self):

print(f"{self.make} {self.model}'s engine has started.")

# Creating an object (instance of the class)

my_car = Car("Toyota", "Corolla", 2020)

# Accessing object attributes and methods

print(my_car.make) # Output: Toyota

my_car.start_engine() # Output: Toyota Corolla's engine has started.

2. Inheritance

Inheritance allows one class (child class) to inherit attributes and methods
from another class (parent class). This helps reuse code and create
hierarchical relationships.

Example:

python

Copy code

# Base class (Parent)

class Animal:

def __init__(self, name):

self.name = name

def make_sound(self):

print(f"{self.name} makes a sound.")

# Derived class (Child)

class Dog(Animal):

def make_sound(self): # Overriding the method in the base class

print(f"{self.name} barks.")
# Creating instances of the classes

animal = Animal("Generic Animal")

dog = Dog("Buddy")

animal.make_sound() # Output: Generic Animal makes a sound.

dog.make_sound() # Output: Buddy barks.

Here, the Dog class inherits the properties of the Animal class and overrides
the make_sound() method to provide its own functionality.

3. Polymorphism

Polymorphism allows objects of different classes to be treated as objects of a

common base class. It means the same method can behave differently based
on the object that calls it.

Example:

python

Copy code

class Bird:

def make_sound(self):

print("Chirp")

class Cat:

def make_sound(self):

print("Meow")

# Polymorphism with a function

def animal_sound(animal):
animal.make_sound()

# Objects of different classes

bird = Bird()

cat = Cat()

# Both objects respond to the same method

animal_sound(bird) # Output: Chirp

animal_sound(cat) # Output: Meow

In this example, both Bird and Cat have the make_sound() method, but they
behave differently. The animal_sound() function can call make_sound() on
any object with that method, regardless of the class.

4. Encapsulation

Encapsulation refers to the bundling of data and methods that operate on

the data into a single unit (class), and restricting direct access to some of
the object's components. This is achieved using private variables and
methods (using underscore _ or double underscore __).

Example:

python

Copy code

class BankAccount:

def __init__(self, balance):

self.__balance = balance # Private attribute

def deposit(self, amount):

self.__balance += amount
def withdraw(self, amount):

if amount <= self.__balance:

self.__balance -= amount

else:

print("Insufficient balance")

def get_balance(self):

return self.__balance

# Creating an instance of BankAccount

account = BankAccount(1000)

# Accessing the balance through a method (Encapsulation)

account.deposit(500)

account.withdraw(300)

print(account.get_balance()) # Output: 1200

# Trying to access the private variable directly (will raise an error)

# print(account.__balance) # AttributeError: 'BankAccount' object has no

attribute '__balance'

Here, the __balance attribute is encapsulated within the class. It can only
be accessed or modified through methods, ensuring controlled access to the
data.

5. Summary of OOP Concepts:

• Class: A blueprint for creating objects. It defines attributes and
methods.
• Object: An instance of a class. It can hold data and interact with
methods defined in the class.
• Inheritance: One class can inherit attributes and methods from
another, promoting code reuse.
• Polymorphism: The same method name can have different
implementations in different classes, depending on the object.
• Encapsulation: Restrict access to certain data or methods to protect
the integrity of an object's data.

3.2. Applying python Date and time concepts

1. Description of Date and Time Libraries

datetime Module (Built-in)

• The datetime module provides basic classes for manipulating dates

and times, including date arithmetic and formatting.

dateutil Module (Third-Party)

• dateutil extends the functionality of datetime and provides powerful

features for parsing, calculating relative deltas, and handling time
zones.

arrow Module (Third-Party)

• arrow is a library that simplifies date and time manipulation by

providing easy-to-use, human-friendly syntax.

pendulum Module (Third-Party)

• pendulum is similar to arrow but also handles time zones and periods
efficiently. It is designed to be more intuitive and consistent.

python-tzdata (Time Zone Database)

• This package provides time zone support, which can be used with
other libraries like datetime, arrow, and pendulum.

2. Set Time Zones

Using datetime:

The built-in datetime module can be used with pytz or zoneinfo to manage
time zones.
python

Copy code

from datetime import datetime

import pytz

# Get the current time in UTC

utc_time = datetime.now(pytz.utc)

print("UTC Time:", utc_time)

# Convert UTC to a specific time zone

local_tz = pytz.timezone('Asia/Kolkata')

local_time = utc_time.astimezone(local_tz)

print("Local Time (Kolkata):", local_time)

Using arrow:

Arrow simplifies time zone handling.

python

Copy code

import arrow

# Get the current time in a specific time zone

local_time = arrow.now('America/New_York')

print("Local Time (New York):", local_time)

# Convert to another time zone

tokyo_time = local_time.to('Asia/Tokyo')
print("Tokyo Time:", tokyo_time)

Using pendulum:

Pendulum automatically handles time zones and provides a clear API for
conversions.

python

Copy code

import pendulum

# Get the current time in a specific time zone

ny_time = pendulum.now('America/New_York')

print("New York Time:", ny_time)

# Convert to another time zone

tokyo_time = ny_time.in_timezone('Asia/Tokyo')

print("Tokyo Time:", tokyo_time)

3. Formatting and Parsing Dates

Using datetime:

You can format and parse dates using strftime() and strptime().

python

Copy code

from datetime import datetime

# Formatting: Convert datetime object to string

now = datetime.now()
formatted_time = now.strftime('%Y-%m-%d %H:%M:%S')

print("Formatted Time:", formatted_time)

# Parsing: Convert string to datetime object

date_string = '2024-09-08 14:30:00'

parsed_date = datetime.strptime(date_string, '%Y-%m-%d %H:%M:%S')

print("Parsed Date:", parsed_date)

Using arrow:

Arrow makes parsing and formatting more straightforward.

python

Copy code

import arrow

# Parsing a date string

parsed_date = arrow.get('2024-09-08 14:30:00', 'YYYY-MM-DD HH:mm:ss')

print("Parsed Date:", parsed_date)

# Formatting a date

formatted_date = parsed_date.format('YYYY-MM-DD HH:mm:ss')

print("Formatted Date:", formatted_date)

Using pendulum:

Pendulum also provides an easy way to parse and format dates.

python

Copy code

import pendulum
# Parsing a date string

dt = pendulum.parse('2024-09-08 14:30:00')

print("Parsed Date:", dt)

# Formatting a date

formatted = dt.format('YYYY-MM-DD HH:mm:ss')

print("Formatted Date:", formatted)

4. Performing Relative Timedeltas

Using datetime:

The timedelta class can be used to perform relative date and time
operations.

python

Copy code

from datetime import datetime, timedelta

# Get the current time

now = datetime.now()

# Add 7 days to the current time

future_date = now + timedelta(days=7)

print("7 Days Later:", future_date)

# Subtract 1 hour from the current time

past_time = now - timedelta(hours=1)

print("1 Hour Ago:", past_time)

Using dateutil.relativedelta:

dateutil.relativedelta allows more flexible date operations like adding months

or years.

python

Copy code

from datetime import datetime

from dateutil.relativedelta import relativedelta

# Get the current time

now = datetime.now()

# Add 1 year and 2 months

future_date = now + relativedelta(years=1, months=2)

print("1 Year, 2 Months Later:", future_date)

Using arrow:

Arrow also allows for relative time deltas.

python

Copy code

import arrow

# Add 5 days

future_date = arrow.now().shift(days=5)

print("5 Days Later:", future_date)

# Subtract 3 hours

past_time = arrow.now().shift(hours=-3)

print("3 Hours Ago:", past_time)

Using pendulum:

Pendulum has built-in support for relative time deltas using the add() or
subtract() methods.

python

Copy code

import pendulum

# Get the current time

now = pendulum.now()

# Add 2 weeks and subtract 3 days

new_time = now.add(weeks=2).subtract(days=3)

print("2 Weeks Later, 3 Days Earlier:", new_time)

5. Summary of Date and Time Concepts:

• Set Time Zones: Manage time zones easily using pytz, arrow, or
pendulum.
• Formatting and Parsing: Use strftime and strptime for string
formatting and parsing in datetime, and use easier functions in arrow
and pendulum.
• Performing Relative Timedeltas: Add or subtract days, months, or
years using timedelta, relativedelta, or similar features in arrow and
pendulum.

3.3. Applying Python Libraries

1. Description of Python Libraries
A Python library is a collection of pre-written code that provides specific
functionality, allowing developers to reuse code rather than writing
everything from scratch. Libraries can handle tasks like data manipulation,
visualization, machine learning, web development, and more. Python
libraries are either part of the standard library or can be installed using
external tools like pip.

2. Python Standard Library

The Python Standard Library includes a wide range of built-in modules

that are installed with Python. These modules provide functionality for
common tasks like file I/O, string handling, mathematics, networking, and
much more.

Examples:

• os: Interacting with the operating system.

• sys: Access system-specific parameters and functions.
• datetime: Work with dates and times.
• math: Perform mathematical functions like trigonometry, logarithms,
etc.

3. Popular Third-Party Libraries

1. Matplotlib:

• A popular plotting library for creating static, interactive, and animated

visualizations in Python.

2. NumPy:

• Provides support for large, multi-dimensional arrays and matrices,

along with a large collection of mathematical functions to operate on
these arrays.

3. Pandas:

• A data manipulation and analysis library, especially useful for

working with structured data like dataframes.

4. Use of Libraries

Importing Libraries

To use any library in Python, whether from the standard library or third-
party, you need to import it using the import statement.
python

Copy code

# Importing the entire library

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

# Importing specific functionality from a library

from math import sqrt, pi

Accessing Functionality

Once a library is imported, you can access its functions, classes, and
methods using dot notation.

Example with NumPy:

python

Copy code

import numpy as np

# Creating a NumPy array

array = np.array([1, 2, 3, 4])

# Performing mathematical operations on the array

mean_value = np.mean(array)

print("Mean Value:", mean_value)

Example with Pandas:

python
Copy code

import pandas as pd

# Creating a Pandas DataFrame

data = {'Name': ['Alice', 'Bob', 'Charlie'], 'Age': [25, 30, 35]}

df = pd.DataFrame(data)

# Accessing functionality: Display first rows of the DataFrame

print(df.head())

Example with Matplotlib:

python

Copy code

import matplotlib.pyplot as plt

# Creating a simple line plot

x = [0, 1, 2, 3]

y = [0, 1, 4, 9]

plt.plot(x, y)

# Adding labels and title

plt.xlabel('X-axis')

plt.ylabel('Y-axis')

plt.title('Line Plot Example')

# Display the plot

plt.show()

Understanding Scope According to Namespace

When you import libraries or functions, they reside in a namespace, which

helps organize and avoid conflicts between function or variable names.

• Global Scope: Variables and functions are accessible from anywhere

in the script.
• Local Scope: Variables and functions are accessible only within a
specific function or block.
• Module Scope: When you import a module, all the module’s functions
and variables are part of that module's namespace.

Example of using aliasing to avoid name conflicts:

python

Copy code

import numpy as np

import pandas as pd

# Using the alias np for NumPy

array = np.array([1, 2, 3, 4])

# Using the alias pd for Pandas

df = pd.DataFrame(array, columns=['Numbers'])

print(df)

Aliasing helps create a clear distinction between different libraries,

especially when they have functions or methods with similar names.

5. Summary of Key Concepts

• Description of Python Library: Libraries are pre-written code

collections that provide ready-to-use functionality.
• Python Standard Library: Built-in modules like os, math, and
datetime for common programming tasks.
• Popular Third-Party Libraries:
o Matplotlib for data visualization.
o NumPy for array/matrix operations and numerical
computation.
o Pandas for data manipulation and analysis.
• Use of Libraries: Import libraries using import and access
functionality using dot notation. Manage scope and avoid name
conflicts with aliasing and namespaces.

3.4. Automation of application post deployment

tasks
Here's a comprehensive guide to automating application post-deployment
tasks using Python, focusing on automating key tasks like database
migrations, configuration file updates, service restarts, and more. The
process includes prioritizing tasks, selecting appropriate Python automation
libraries, developing Python scripts, integrating them with deployment
processes, and finally, testing and monitoring.

1. Identification of Tasks to Automate

Post-deployment tasks vary depending on the application and environment

but often include:

• Database migrations: Applying schema updates or versioning.

• Configuration file updates: Modifying or adding environment-specific
configurations.
• Service restarts: Restarting or reloading system services to apply new
configurations.
• Testing and verification: Running health checks or smoke tests to
ensure the deployment was successful.
• Logging and notifications: Capturing logs and sending notifications
(e.g., via Slack or email) when tasks are complete.

2. Identification of Tasks to Be Prioritized

When deciding which tasks to automate, prioritize based on:

• Repetitive: Tasks that are repeated every deployment, such as

database migrations.
• Time-consuming: Tasks that take longer than desired, like
configuration file updates.
• Error-prone: Tasks that are manual and prone to human error, such
as service restarts.
• Critical for deployment speed: Tasks that are crucial for successful
and fast rollouts, such as automated tests and service health checks.
3. Selection of a Python Automation Library

Several Python libraries can automate post-deployment tasks. Choosing the

right one depends on the environment, infrastructure, and complexity of the
tasks.

Popular Python Automation Libraries:

• Fabric: Simplifies the automation of SSH-based tasks. Ideal for

managing servers, running commands, or file manipulation.

Code:

pip install fabric

• Ansible: Uses declarative YAML playbooks for automating

configuration, service management, and deployment. Can be
controlled via Python.

Code:

pip install ansible

• SaltStack: Scalable and flexible, ideal for automating infrastructure

tasks, especially in large server environments.
• Boto3: AWS SDK for Python, used to automate tasks like scaling
infrastructure, deploying code, and updating configurations in AWS
environments.

Code:

pip install boto3

• vSphere Automation SDK for Python: Ideal for automating VMware

vSphere environments, managing virtual machines, networks, and
storage.

4. Develop the Python Script

Once the appropriate library is selected, you can start developing the
automation script. Here's an example using Fabric to automate service
restarts, database migrations, and file updates:

Example Python Automation Script Using Fabric:

python
Copy code

from fabric import Connection, task

# Database migration task

@task

def migrate_db(c):

c.run('cd /path/to/app && python manage.py migrate')

print("Database migrations applied successfully.")

# Update configuration files

@task

def update_configs(c):

c.put('/local/path/config.ini', '/remote/path/config.ini')

print("Configuration file updated.")

# Restart services

@task

def restart_services(c):

c.run('sudo systemctl restart nginx')

c.run('sudo systemctl restart app-service')

print("Services restarted.")

# Main automation workflow

@task

def post_deploy(c):
print("Starting post-deployment tasks...")

migrate_db(c)

update_configs(c)

restart_services(c)

print("All post-deployment tasks completed.")

In this example:

• migrate_db: Runs the database migration using Django's manage.py.

• update_configs: Copies a local config file to the remote server.
• restart_services: Restarts Nginx and the application services.

Structuring Logically and Logging:

python

Copy code

import logging

# Setup logging

logging.basicConfig(filename='post_deploy.log', level=logging.INFO)

# Log the migration task

def migrate_db(c):

c.run('cd /path/to/app && python manage.py migrate')

logging.info("Database migrations applied successfully.")

Here, all actions and outcomes are logged for future reference and
debugging.

5. Integrate Script with Deployment Process

There are several ways to trigger your automation script after the
deployment:

1. Direct Execution After Deployment Completion:

• The script can be run immediately after deployment by adding it to the
final step of your deployment pipeline or server hooks.

Example

Copy code

python3 post_deploy.py

2. Integration with CI/CD Pipelines:

• In CI/CD pipelines like Jenkins, GitLab CI, or GitHub Actions, you

can trigger the script at a post-deployment stage.

Example with GitLab CI:

yaml

Copy code

deploy:

script:

- ./deploy_script.sh

after_script:

- python3 post_deploy.py # Trigger post-deployment automation

3. Scheduled Execution:

• You can also schedule the script using cron jobs to run at specific
intervals.

Example cron job:

bash

Copy code

0 3 * * * /usr/bin/python3 /path/to/post_deploy.py

4. Security Measures:

• Access Control: Limit access to the script by ensuring only

authorized users or processes can execute it.
• Environment Variables: Store sensitive information (like API keys) in
environment variables instead of hardcoding them in the script.
• Use Encryption: Secure files and communication (e.g., SSH keys, API
calls) to avoid exposing sensitive data.

6. Testing and Monitoring

Thorough Testing:

• Test each function (e.g., database migrations, service restarts)

separately in a staging environment to ensure they work as expected.

Monitor Script Logs:

• Regularly check the logs to ensure tasks are being executed

successfully. Set up automated alerts if tasks fail or if certain
thresholds (e.g., execution time) are breached.

Refining and Improving:

• Continuously improve the script based on feedback from logs and

error reports. For example, add more detailed logging or exception
handling.

Example of Exception Handling in the Script:

python

Copy code

import logging

def restart_services(c):

try:

c.run('sudo systemctl restart nginx')

c.run('sudo systemctl restart app-service')

logging.info("Services restarted successfully.")

except Exception as e:

logging.error(f"Failed to restart services: {e}")

Summary:
1. Identify Tasks: Automate tasks like database migrations,
configuration updates, and service restarts.
2. Prioritize Tasks: Focus on repetitive, error-prone, and time-
consuming tasks.
3. Select a Library: Choose from Fabric, Ansible, Boto3, etc., based on
your environment.
4. Develop the Script: Use the library’s functionality to automate the
tasks logically and securely.
5. Integrate with Deployment: Trigger the script via CI/CD pipelines,
direct execution, or scheduling.
6. Testing and Monitoring: Ensure thorough testing and continuous
monitoring to improve reliability.