Pytorch Tensors

Introduction

The advancement of interest in Deep Learning in recent years and the explosion of
Machine Learning tools like TensorFlow, PyTorch, etc., will also be cited, which will
provide ease of use and easy debugging of codes.

Many popular frameworks such as MxNet, Tensorflow, Jax, PaddlePaddle, Caffe 2,

Mindspore, and Theano will gain popularity because they will construct a static dataflow
graph that represents the computation and that can be applied to batches of data.
Although they will provide visibility into the whole computation and theoretically, they
are leveraged with improved performance and scalability. This will come at the cost of
flexibility, ease of debugging, and ease of use.

Tensor
Tensor is a fundamental building block of Pytorch, and it is basically the same as
a Numpy array. It is mostly used for converting Images, Audio into a mathematical form
used for processing, as computers don’t understand images but standard numbers. Hence,
it is important to convert images into numerical forms.
One of the important features which are offered by tensors is they can store track of all
the operations performed on them, which helps to compute the optimized output; this can
be done by using the Autograd functionality of a tensor.
In simplistic terms, scaler – vector – matrices – tensor as flow
1. Scaler is a 0-dimensional Vector.
2. Vector is a 1-dimensional Vector.
3. Matrices are 2-dimensional Vectors.
4. Tensors are generalized N-dimensional Tensors.

Deep Learning with PyTorch | An Introduction

PyTorch in a lot of ways behaves like the arrays we love from Numpy. These Numpy
arrays, after all, are just tensors.
PyTorch takes these tensors and makes it simple to move them to GPUs for the faster
processing needed when training neural networks.
It also provides a module that automatically calculates gradients (for backpropagation)
and another module specifically for building neural networks.
All together, PyTorch ends up being more flexible with Python and the Numpy stack
compared to TensorFlow and other frameworks.
Neural Networks: Deep Learning is based on artificial neural networks which have been
around in some form since the late 1950s.
The networks are built from individual parts approximating neurons, typically called
units or simply “neurons.” Each unit has some number of weighted inputs.

These weighted inputs are summed together (a linear combination) then passed through
an activation function to get the unit’s output.

Below is an example of a simple

PyTorch is an open-source machine learning library based on the Torch library,
developed by Facebook’s AI Research lab.
It is widely used in deep learning, natural language processing, and computer vision
applications. PyTorch provides a dynamic computational graph, which allows for easy
and efficient modeling of complex neural networks.

Deep Learning is based on artificial neural networks which have been around in some
form since the late 1950s. The networks are built from individual parts approximating
neurons, typically called units or simply “neurons.” Each unit has some number of
weighted inputs. These weighted inputs are summed together (a linear combination) then
passed through an activation function to get the unit’s output. Below is an example of a
simple
PyTorch is an open-source machine learning library based on the Torch library, developed
by Facebook’s AI Research lab. It is widely used in deep learning, natural language
processing, and computer vision applications. PyTorch provides a dynamic computational
graph, which allows for easy and efficient modeling of complex neural networks.
Here’s a general overview of how to use PyTorch for deep learning:
Define the model: PyTorch provides a wide range of pre-built neural network
architectures, such as fully connected networks, convolutional neural networks (CNNs),
and recurrent neural networks (RNNs). The model can be defined using PyTorch’s
torch.nn module.
Define the loss function: The loss function is used to evaluate the model’s performance
and update its parameters. PyTorch provides a wide range of loss functions, such as cross-
entropy loss and mean squared error loss.
Define the optimizer: The optimizer is used to update the model’s parameters based on the
gradients of the loss function. PyTorch provides a wide range of optimizers, such as Adam
and SGD.
Train the model: The model is trained on a dataset using the defined loss function and
optimizer. PyTorch provides a convenient data loading and processing library,
torch.utils.data, that can be used to load and prepare the data.
Evaluate the model: After training, the model’s performance can be evaluated on a test
dataset.
Deploy the model: The trained model can be deployed in a wide range of applications,
such as computer vision and natural language processing.
neural net.

Tensors: It turns out neural network computations are just a bunch of linear algebra
operations on tensors, which are a generalization of matrices. A vector is a 1-dimensional
tensor, a matrix is a 2-dimensional tensor, an array with three indices is a 3-dimensional
tensor. The fundamental data structure for neural networks are tensors and PyTorch is built
around tensors. It’s time to explore how we can use PyTorch to build a simple neural
network.

 Python3

# First, import PyTorch

import torch

Define an activation function(sigmoid) to compute the linear output

 Python3

def activation(x):
""" Sigmoid activation function

Arguments

---------

x: torch.Tensor

"""

return 1/(1 + torch.exp(-x))

 Python3

# Generate some data

# Features are 3 random normal variables

features = torch.randn((1, 5))

# True weights for our data, random normal variables again

weights = torch.randn_like(features)

# and a true bias term

bias = torch.randn((1, 1))

ADVANTAGES AND DISADVANTAGES:

Advantages of using PyTorch for deep learning:

Dynamic computational graph: PyTorch’s dynamic computational graph allows for

easy and efficient modeling of complex neural networks. This makes it well-suited for
tasks such as natural language processing and computer vision.
User-friendly API: PyTorch has a user-friendly and intuitive API, which makes it easy
to implement, debug, and experiment with different models.
Active community: PyTorch has a large and active community, which means that you
can find a lot of tutorials, pre-trained models, and other resources to help you get started.
Integration with other libraries: PyTorch has great integration with other libraries such
as CUDA, which allows for fast and efficient training of models on GPUs.
Easy to use for distributed training: PyTorch makes it easy to perform distributed
training across multiple GPUs and machines, which can significantly speed up training
time.

Disadvantages of using PyTorch for deep learning:

Less mature than TensorFlow: PyTorch is relatively new compared to TensorFlow,

which means that it may have less mature libraries and fewer resources available.
Less production ready: PyTorch is more suited for research and development work,
rather than for production-level deployment.
Less optimized for mobile: PyTorch does not have as good support for mobile
deployment as TensorFlow Lite.
Less optimized for browser: PyTorch does not have as good support for browser
deployment as TensorFlow.js
Less optimized for Autograd: PyTorch’s Autograd implementation is less optimized
than TensorFlow’s, which can make training large models more memory-intensive.
CNN in Pytorch
A Convolutional Neural Network (CNN) is a type of Deep Learning neural network
architecture commonly used in Computer Vision. Computer vision is a field of Artificial
Intelligence that enables a computer to understand and interpret the image or visual
data.
When it comes to Machine Learning, Artificial Neural Networks perform really well.
Neural Networks are used in various datasets like images, audio, and text. Different
types of Neural Networks are used for different purposes, for example for predicting the
sequence of words we use Recurrent Neural Networks more precisely an LSTM,
similarly for image classification we use Convolution Neural networks. In this blog, we
are going to build a basic building block for CNN.
In a regular Neural Network there are three types of layers:
1. Input Layers: It’s the layer in which we give input to our model. The number
of neurons in this layer is equal to the total number of features in our data
(number of pixels in the case of an image).
2. Hidden Layer: The input from the Input layer is then fed into the hidden
layer. There can be many hidden layers depending on our model and data size.
Each hidden layer can have different numbers of neurons which are generally
greater than the number of features. The output from each layer is computed
by matrix multiplication of the output of the previous layer with learnable
weights of that layer and then by the addition of learnable biases followed by
activation function which makes the network nonlinear.
3. Output Layer: The output from the hidden layer is then fed into a logist ic
function like sigmoid or softmax which converts the output of each class into
the probability score of each class.
The data is fed into the model and output from each layer is obtained from the above step
is called feedforward, we then calculate the error using an error function, some common
error functions are cross-entropy, square loss error, etc. The error function measures how
well the network is performing. After that, we backpropagate into the model by
calculating the derivatives. This step is called Backpropagation which basically is used
to minimize the loss.
Installation
For the implementation of the CNN and downloading the CIFAR-10 dataset, we’ll be
requiring the torch and torchvision modules. Apart from that, we’ll be using numpy and
matplotlib for data analysis and plotting. The required libraries can be installed using the
pip package manager through the following command:
Stepwise implementation
Downloading data and printing some sample images from the training set.
 Before starting our journey to implementing CNN, we first need to download
the dataset onto our local machine, which we’ll be training our model over.
We’ll be using the torchvision utility for this purpose and downloading the
CIFAR-10 dataset into training and testing sets in
directories “./CIFAR10/train” and “./CIFAR10/test,“ respectively. We also
apply a normalized transform where the procedure is done over the three
channels for all the images.
 Now, we have a training dataset and a test dataset with 50000 and 10000
images, respectively, of a dimension 32x32x3. After that, we convert these
datasets into data loaders of a batch size of 128 for better generalization and a
faster training process.
 Finally, we plot out some sample images from the 1st training batch to get an
idea of the images we’re dealing with using the make_grid utility
from torchvision.

Implementing the CNN architecture

On the architecture side, we’ll be using a simple model that employs three convolution
layers with depths 32, 64, and 64, respectively, followed by two fully connected layers
for performing classification.
 Each convolutional layer involves a convolutional operation involving a 3×3
convolution filter and is followed by a ReLU activation operation for
introducing nonlinearity into the system and a max-pooling operation with a
2×2 filter to reduce the dimensionality of the feature map.
 After the end of the convolutional blocks, we flatten the multidimensional
layer into a low dimensional structure for starting our classification blocks.
After the first linear layer, the last output layer(also a linear layer) has ten
neurons for each of the ten unique classes in our dataset.
The architecture is as follows:

Figure 3: Architecture of the CNN

For building our model, we’ll make a CNN class inherited from
the torch.nn.Module class for taking advantage of the Pytorch utilities. Apart from that,
we’ll be using the torch.nn.Sequential container to combine our layers one after the
other.
 The Conv2D(), ReLU(), and MaxPool2D() layers perform the convolution,
activation, and pooling operations. We used padding of 1 to give sufficient
learning space to the kernel as padding gives the image more coverage area,
especially the pixels in the outer frame.
 After the convolutional blocks, the Linear() fully connected layers perform
classification.