UNIX SHELL PROGRAMMING Micro Complete
UNIX SHELL PROGRAMMING Micro Complete
UNIX is a powerful Operating System initially developed by Ken Thompson, Dennis Ritchie at AT&T
Bell laboratories in 1970.
The Unix operating system is a set of programs that act as a link between the computer and the user.
A UNIX operating system is a multitasking operating system that allows you to initiate more than
one task from the same terminal so that one task is performed as a foreground and the other task as
a background process.
Advantages of UNIX operating system
Ecommerce: - Many big online stores use UNIX or Linux servers to host their websites. UNIX has also
used to manage mobiles and electronic machines.
Memory usage: - UNIX use less memory while running sophisticated programs. UNIX OS can handle
virtual memory nicely. The virtual memory expands as more programs come into the main memory.
Most of the tasks in UNIX is done by using fewer resources.
Less code to execute: - In GUI (graphical user interface), we sometimes need many mouse clicks to
perform some specific task but in the case of UNIX we can simply write one command in CLI
(command line interface) and that task is done.
Files used everywhere: - All type of data is stored in files i.e. all devices and terminal are stored in
files. Working with files in OS becomes fast and can be easily managed by UNIX.
No anti-virus needed: - As you know that Chrome OS, macOS, Linux, Ubuntu and android are
developed by using UNIX OS. These OS are considered safe from any virus. You don’t need to install
any anti-virus in newly build Chrome OS.
User interactions: - There are many online servers, electric machines where user interaction is not
involved. UNIX is an ideal choice for machines and devices where no or fewer user interactions are
involved. UNIX can manage the automatic working of systems very well.
Safe and secure: - UNIX provides a safe and secure platform in which multiple users can interact with
the servers online without any security issues. The interaction with the UNIX servers is fast and
without any bugs.
Linux OS Unix OS
It has different distros like Ubuntu, Redhat, Fedora, etc It has also different distros like IBM AIX,
HP-UX and Sun Solaris.
Nowadays, Linux is in great demand. Anyone can use It was developed mainly for servers,
Linux whether a home user, developer or a student. workstations and mainframes.
Linux is used everywhere from servers, PC, smartphones, It is used in servers, workstations and PCs.
tablets to mainframes and supercomputers.
Linux supports more file system than Unix. It also supports file system but lesser than
Linux.
Linux is a Unix clone,behaves like Unix but doesn't Unix contain a completely different coding
contain its code. developed by AT&T Labs.
UNIX OS Windows OS
It has a text base interface, making it harder to grasp for It has a Graphical User Interface,
newcomers. making it simpler to use.
It is tedious to create a backup and recovery system in It has an integrated backup and
UNIX, but it is improving with the introduction of new recovery system that make it simpler
distributions of Unix. to use.
Hardware support is limited in UNIX system. Some Drivers are available for almost all the
hardware might not have drivers built for them. hardware.
CREATION OF PARTITION IN OS
Partitioning a disk in Unix is a process of dividing a disk into one or more logical areas, known as
partitions, on which the user can work separately . The partition table stores the information about
the location and size of partitions. With the partition table, each partition can appear to the operating
system as a logical disk, and users can read and write data on those disks.
Here are the steps to create partitions in Unix:
1. Attach the disk to the proper port.
2. Create partitions in the disk.
3. Create a file system on the partition.
4. Mount the file systems.
To create a new partition, you can use the `fdisk` command with the name of the hard drive
you want to create partitions on . Once you enter the command mode, you can create a new partition
using the `n` command . After creating the partition, you can set the partition type using the `t`
command . Finally, you can save the changes using the `w` command .
Once you have created the partition, you can create a file system on the partition using the
`mkfs` command . For example, to create an ext4 file system on a specific partition, you can use the
following command: `sudo mkfs.ext4 /dev/sda1` .
After creating the file system, you can mount the file system using the `mount` command . For example,
to mount the file system on `/mnt/data`, you can use the following command: `sudo mount /dev/sda1
/mnt/data` .
PROCESSES & ITS CREATION PHASES – FORK , EXEC, WAIT, EXIT
In Unix, a process is an instance of a program that is being executed. When a command is issued
in Unix/Linux, it creates/starts a new process 1. The newly created process is called the child process,
and the process that initiated it (or the process when execution is started) is called the parent process .
Fork : The creation of a process in Unix is achieved through the fork() system call 2. The fork()
system call creates a child process that is a copy of the current task 3. After the fork() system
call, we have two processes: the parent and child processes.
Exec : The exec() system call is used to replace the current process image with a new process
image . The exec() system call is used after the fork() system call to execute a new program in
the child process . The exec() system call replaces the current process image with a new
process image, which is loaded from the executable file.
wait : The wait() system call is used to wait for the child process to terminate 5. The wait()
system call suspends the execution of the parent process until the child process terminates 5.
exit : The exit() system call is used to terminate the current process . The exit() system call is
used to terminate the current process and return the exit status to the parent process .
GRANTING RIGHTS
In Unix, file permissions are used to control access to files and directories. There are three types
of permissions: read, write, and execute. These permissions can be granted to three types of users:
owner, group, and others . To grant permissions to a file or directory, we can use the chmod
command followed by the permission code and the file or directory name.
The permission code is a three-digit number, where each digit represents the permission for each user
type. The first digit represents the permission for the owner, the second digit represents the
permission for the group, and the third digit represents the permission for others.
The permission codes are as follows:
4 for read permission
2 for write permission
1 for execute permission
For example, to grant read, write, and execute permissions to the owner, read and execute
permissions to the group, and read permission to others, you can use the following command:
chmod 751 filename
Here, the first digit (7) represents the permission for the owner, which is read, write, and execute.
The second digit (5) represents the permission for the group, which is read and execute. The third
digit (1) represents the permission for others, which is read only .
User management commands :
UNIX COMMAND - Unix commands are a set of commands that are used to interact with the Unix
operating system. Unix commands are entered at the command prompt in a terminal window, and
they allow users to perform a wide variety of tasks, such as managing files and directories, running
processes, managing user accounts, and configuring network settings.
TYPE OF USER MANAGEMENT COMMANDS -
useradd: This command is used to create a new user account. For example, to create a new user
named “johndoe”, you can use the following command: sudo useradd johndoe.
userdel: This command is used to delete an existing user account. For example, to delete the user
account “johndoe”, you can use the following command: sudo userdel johndoe.
passwd: This command is used to change the password of an existing user account. For example, to
change the password for the user account “johndoe”, you can use the following command: sudo
passwd johndoe.
usermod: This command is used to modify an existing user account. For example, to change the home
directory of the user account “johndoe”, you can use the following command: sudo usermod -d
/new/home/directory johndoe.
su: This command is used to switch to another user account. For example, to switch to the user
account “johndoe”, you can use the following command: su johndoe.
Unix Architecture
The following table lists out commands that are available on majority of Unix systems to create
and manage accounts and groups −
Sr.No. Command & Description
DISK QUOTA
File quota is a built-in feature found in the Linux kernel that determines the amount of space a file
should have to support user activities. The disk quotas also limit the number of files a user can create
on the system. File systems that support the quota system include ext2, ext3, ext4, and XFS .
SHELL
SHELL is a program which provides the interface between the user and an operating system. When
the user logs in OS starts a shell for user. Kernel controls all essential computer operations, and
provides the restriction to hardware access, coordinates all executing utilities, and manages
Resources between process. Using kernel only user can access utilities provided by operating
system.
Types of Shell in UNIX
The Bourne Shell (sh)
Introduced with Version 7 UNIX in 1979, the Bourne Shell (often denoted as sh) has become
synonymous with UNIX Shell scripting. This pioneering Shell offered users a programming
environment, complete with variables, control structures, and built-in operators. Its simplicity was its
strength, ensuring widespread adoption in UNIX environments.
As the foundational Shell, sh paved the way for the development of subsequent Shells. Its scripts are
characterised by portability across various UNIX-based systems, making it the go-to choice for many
legacy applications even today.
The C Shell (csh)
Emerging from the University of California, Berkeley in the late 1970s, the C Shell (csh) aimed to offer
an improved User Experience. Notably, its syntax mirrors that of the C programming language,
providing an intuitive environment for those familiar with C. Enhanced scripting features, including
job control and command history, make it quite interactive.
However, csh has faced criticism for certain scripting quirks and behaviours, making it less favoured
for scripting tasks compared to other Shells. Despite this, for interactive use, many found its features
compelling.
The Korn Shell (ksh)
David Korn's brainchild in the early 1980s, the Korn Shell (ksh), brought the best of both
the Bourne and C Shells. Infusing the simplicity of sh with advanced features of csh, ksh introduced
associative arrays and advanced string manipulation capabilities.
The Korn Shell’s adaptability has seen it become the default Shell for many commercial UNIX systems.
Over the years, ksh has enjoyed several upgrades, each refining its capabilities and improving User
Experience. For scriptwriters seeking a balanced Shell, ksh often stands out as an optimal choice.
Unlock the power of UNIX with our UNIX Fundamentals Course – join today!
The Bourne Again Shell (bash)
A product of the GNU project in 1989, the Bourne Again Shell (bash) sought to enhance
the Bourne Shell experience. Incorporating features from both ksh and csh, bash presents one of the
most comprehensive and user-friendly Shell environments.
The Bourne Again Shell’s ubiquity in the Linux realm is testament to its versatility and power. Being
the default Shell for most Linux distributions and even macOS, it offers users a consistent interface
packed with advanced features. From scripting to interactive usage, bash caters to both novices and
experts alike.
The Tcsh Shell
An evolution of the C Shell, tcsh surfaced in the late 1980s, addressing many of csh's limitations.
Introducing command-line editing, autocompletion, and improved scripting capabilities, tcsh aimed to
give users a more rounded experience.
The enhancements made the Tcsh Shell a preferred choice for many who appreciated the csh lineage
but sought a richer feature set. While not as prominent as some of the other Shells, tcsh has its loyal
user base, especially among those who began their UNIX journey during its heyday.
The Z Shell (zsh)
A relative newcomer, zsh, was released in 1990, it combined the best of bash, ksh, and tcsh. What
sets zsh apart is its customisability, advanced features such as shared command history, spelling
correction, and themeable prompts. It offers users a tailored Shell experience.
Over the years, zsh has grown in popularity, particularly among developers and power users. This
ascent was further cemented when macOS Catalina chose zsh as its default Shell. Its dynamic
community ensures regular updates, making zsh a compelling choice for modern users.
LINK
A link in UNIX is a pointer to a file. Like pointers in any programming languages, links in UNIX
are pointers pointing to a file or a directory. Creating links is a kind of shortcuts to access a file.
Links allow more than one file name to refer to the same file, elsewhere.
There are two types of links :
1. Soft Link or Symbolic links
2. Hard Links
1. Soft Links / Symbolic Links
A soft link is similar to the file shortcut feature which is used in Windows Operating systems.
Each soft linked file contains a separate Inode value that points to the original file. As similar to
hard links, any changes to the data in either file is reflected in the other. Soft links can be linked
across different file systems, although if the original file is deleted or moved, the soft linked file
will not work correctly (called hanging link).
ls -l command shows all links with first column value l? and the link points to original file.
Soft Link contains the path for original file and not the contents.
Removing soft link doesn’t affect anything but removing original file, the link becomes “dangling”
link which points to nonexistent file.
A soft link can link to a directory.
The size of the soft link is equal to the length of the path of the original file we gave. E.g if we link
a file like ln -s /tmp/hello.txt /tmp/link.txt then the size of the file will be 14bytes which is
equal to the length of the “/tmp/hello.txt”.
If we change the name of the original file then all the soft links for that file become dangling i.e.
they are worthless now.
Command to create a Soft link is:
$ ln -s [original filename] [link name]
2.Hard Links
Each hard linked file is assigned the same Inode value as the original, therefore they reference
the same physical file location. Hard links more flexible and remain linked even if the original or
linked files are moved throughout the file system, although hard links are unable to cross
different file systems.
ls -l command shows all the links with the link column shows number of links.
Links have actual file contents
Removing any link, just reduces the link count, but doesn’t affect other links.
Even if we change the filename of the original file then also the hard links properly work.
We cannot create a hard link for a directory to avoid recursive loops.
If original file is removed then the link will still show the content of the file.
The size of any of the hard link file is same as the original file and if we change the content in any
of the hard links then size of all hard link files are updated.
The disadvantage of hard links is that it cannot be created for files on different file systems and it
cannot be created for special files or directories.
Command to create a hard link is:
$ ln [original filename] [link name]
File permissions
File ownership is an important component of Unix that provides a secure method for storing files.
Every file in Unix has the following attributes −
Owner permissions − The owner's permissions determine what actions the owner of the file
can perform on the file.
Group permissions − The group's permissions determine what actions a user, who is a
member of the group that a file belongs to, can perform on the file.
Other (world) permissions − The permissions for others indicate what action all other users
can perform on the file.
SYSTEM CALL
A system call is a procedure that provides the interface between a process and the operating
system. It is the way by which a computer program requests a service from the kernel of the
operating system.
System calls are divided into 5 categories mainly :
Process management system calls: These system calls provide a way to create, manage, and control
processes. Examples of process management system calls include fork(), exec(), wait(), and exit() .
File management system calls: These system calls provide a way for programs to interact with the
file system. Examples of file management system calls include open(), read(), write(), and close() .
Device management system calls: These system calls provide a way for programs to interact with
devices such as printers, disks, and network interfaces. Examples of device management system calls
include ioctl(), read(), and write() .
Network management system calls: These system calls provide a way for programs to interact with
the network stack. Examples of network management system calls include socket(), bind(), listen(),
and accept() .
System information system calls: These system calls provide a way for programs to obtain
information about the system. Examples of system information system calls include getpid(), getuid(),
and gettimeofday() .
Look at the above snapshot, script echo Hello World is created with echo command
as hello_world. Now command chmod +x hello_world is passed to make it executable. We have
given the command ./hello_world to mention the hello_world path. And output is displayed.
SHELL VARIABLES
When a shell is running, three main types of variables are present −
Local Variables − A local variable is a variable that is present within the current instance of the shell.
It is not available to programs that are started by the shell. They are set at the command prompt.
Environment Variables − An environment variable is available to any child process of the shell.
Some programs need environment variables in order to function correctly. Usually, a shell script
defines only those environment variables that are needed by the programs that it runs.
Shell Variables − A shell variable is a special variable that is set by the shell and is required by the
shell in order to function correctly. Some of these variables are environment variables whereas
others are local variables.
In shell scripting, you can define your own variables using the = operator. These variables are
called user-defined variables. They are deleted after the shell script executes .
Shell Variable is used in shell scripts for many functionalities like storing data and
information, taking input from users, printing values that are stored. They are also used for storing
data temporarily and storing output of commands.
Hard Links Soft Links / Symbolic Links
It is a copy of the original file that serves as a It is a short pointer file that links a filename
pointer to the same file, allowing it to be to a pathname. It's nothing more than a shortcut
accessed even if the original file is deleted or to the original file, much like the Windows OS's shortcut
relocated. option.
It has a similar inode number to the target file. It has a different inode number.
It is not allowed the relative path. It allows both relative and absolute paths.
It cannot be established outside the file system. It may be established in the file system.
It is faster. It is slower.
The "ln" command is used to make a hard link The "ln -s" command is used to make a soft link in Linux.
in Linux.
It remains valid even if the target file is deleted. It becomes invalid when the originating file is deleted.
EDITOR
Editors or text editors are software programs that enable the user to create and edit text files. In the field
of programming, the term editor usually refers to source code editors that include many special features for
writing and editing code. Notepad, Wordpad are some of the common editors used on Windows OS and vi,
emacs, Jed, pico are the editors on UNIX OS.
There are several text editors available on UNIX systems -
1. vi: A visual editor that uses ed/ex line-mode commands for global file editing.
2. emacs: A powerful and extensible text editor that is highly customizable .
3. nano: A simple and user-friendly text editor that is easy to use.
4. Sublime Text: A cross-platform text editor that supports multiple programming languages.
5. Atom: A free and open-source text editor that is highly customizable .
VI EDITOR
The default editor that comes with the UNIX operating system is called vi (visual editor). Using vi editor,
we can edit an existing file or create a new file from scratch. we can also use this editor to just read a text file.
The advanced version of the vi editor is the vim editor.
How to open VI editor?
To open vi editors, we just need to type the command mentioned below.
vi [file_name]
Here, [file_name] = this is the file name we want to create or to open the pre-existing file.
Modes of Operation in the vi editor
There are three modes of operation in vi:
1. Command Mode: - When vi starts up, it is in Command Mode. This mode is where vi interprets any
characters we type as commands and thus does not display them in the window. This mode allows us to
move through a file, and delete, copy, or paste a piece of text. Enter into Command Mode from any other
mode, requires pressing the [Esc] key. If we press [Esc] when we are already in Command Mode, then vi
will beep or flash the screen.
2. Insert mode: - This mode enables you to insert text into the file. Everything that’s typed in this mode is
interpreted as input and finally, it is put in the file. The vi always starts in command mode. To enter text,
you must be in insert mode. To come in insert mode, you simply type i. To get out of insert mode, press
the Esc key, which will put you back into command mode.
3. Last Line Mode (Escape Mode): - Line Mode is invoked by typing a colon [:], while vi is in Command
Mode. The cursor will jump to the last line of the screen and vi will wait for a command. This mode
enables you to perform tasks such as saving files and executing commands.
CREATING A USER/ACCOUNT
syntax to create a user's account −
useradd -d homedir -g groupname -m -s shell -u userid accountname
The following table lists out the parameters −
Sr.No. Option & Description
1 -d homedir
Specifies home directory for the account
2 -g groupname
Specifies a group account for this account
3 -m
Creates the home directory if it doesn't exist
4 -s shell
Specifies the default shell for this account
5 -u userid
You can specify a user id for this account
6 accountname
Actual account name to be created
If you do not specify any parameter, then the system makes use of the default values.
The useradd command modifies the /etc/passwd, /etc/shadow, and /etc/group files and creates a
home directory.
Following is the example that creates an account mcmohd, setting its home directory
to /home/mcmohd and the group as developers. This user would have Korn Shell assigned to it.
$ useradd -d /home/mcmohd -g developers -s /bin/ksh mcmohd
Before issuing the above command, make sure you already have the developers group created
using the groupadd command.
Once an account is created you can set its password using the passwd command as follows −
$ passwd mcmohd20
Changing password for user mcmohd20.
New UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
When you type passwd accountname, it gives you an option to change the password, provided
you are a superuser. Otherwise, you can change just your password using the same command but
without specifying your account name.
UNIX FILE SYSTEM
Unix File System is defined as the framework that organizes and stores a large amount of data that can
be handled easily. It consists of elements like a file, a collection of related data that can be viewed
logically as a stream of bytes (or characters). A file is also the smallest storage unit in the Unix file
system.
The file system consists of two main components which are files and directories. The entire file
system follows a hierarchy in which directories act as a special file containing multiple files. The
highest level directory in the entire hierarchical structure is known as the root. The root is denoted as
' / '. There can be many sub-directories under this directory.
FUNCTION
A function is a block of code that performs a specific task and can be called multiple times from
within a script. Functions provide modularity in the program and reduce the code length. They are
useful for performing repetitive tasks and for breaking down the overall functionality of a script into
smaller, logical subsections 12.
To declare a function, simply use the following syntax −
function_name () {
commands
}
Here, function_name is the name of the function, and commands are the commands that the
function will execute.
RECURSIVE FUNCTION - One of the more interesting features of functions is that they can call
themselves and also other functions. A function that calls itself is known as a recursive function.
EXAMPLE
#!/bin/sh
# Define your function here
Hello () {
echo "Hello World"
}
# Invoke your function
Hello
output −
$./test.sh
Hello World
Types of Unix File System
The Unix File Systems are classified into six types, that are:
1. Ordinary Files
The files in Unix, which include program instructions, texts, and data, these files are known as ordinary files.
Some of the characteristics of ordinary files are:
There are no other files in ordinary files.
They are always placed under the directory file.
These files store the information of users. It may be some text or any image which we have to draw.
We mainly work with ordinary files.
2. Directories
Directories store both kinds of files: special as well as ordinary files. Some of the characteristics of
directories are:
Directories may contain other directories also.
All these directories are the descendants of the root directory.
We use directories to organize the collection of files.
It doesn’t contain real information like text or images. PAGE 6
3. Special Files
These files represent physical devices like printers, tape drivers, or a terminal. Some of its
characteristics are:
Special files represent input/output operations on Linux and Unix systems.
There are mainly two special files for every device in the Unix system: block special and character special
files.
Block special: Using it for device input/output will move the data to the higher fixed-size blocks. This kind
of access is also known as block device access.
Character Special: This is one character at a time for the terminal devices. However, in the case of disk
devices, raw access means reading or writing across an entire section of data blocks that are native to our
disk.
4. Pipes
Pipes are used to link commands together to store both kinds. The following are the characteristics of
pipes:
It is a temporary file that only holds the data from one command until another reads it.
Unix pipes provide the one-way flow of data.
Any command's output is assigned to its next command as an input.
5. Sockets
The files that enable advanced inter-process communication are called Sockets. It is also known as an inter-
process communication socket. Some of the characteristics of sockets are:
Unix sockets are used in the framework of client-server applications.
It is essentially a data stream, exactly the same as the network stream and network socket, except where
each transaction is local to the file system.
6. Symbolic Links
It is used to reference another file in the system. Characteristics of symbolic links are:
It involves a text representation of the path to the file it refers to.
To an end-user, a symbolic link will seem to have its name, yet when we need to write or read
data to this file, it will reference these tasks to the file it focuses on.
If the source file is deleted or moved to a different location, the symbolic file will not function.
PASTE
Paste command is one of the useful commands in Unix or Linux operating system. It is used to join
files horizontally (parallel merging) by outputting lines consisting of lines from each file specified,
separated by tab as delimiter, to the standard output. When no file is specified, or put dash (“-“)
instead of file name, paste reads from standard input and gives output as it is until a interrupt
command [Ctrl-c] is given.
Syntax:
paste [OPTION]... [FILES]...
JOIN
The join command in UNIX is a command line utility for joining lines of two files on a common field.
Suppose you have two files and there is a need to combine these two files in a way that the output
makes even more sense.For example, there could be a file containing names and the other
containing ID’s and the requirement is to combine both files in such a way that the names and
corresponding ID’s appear in the same line. join command is the tool for it. join command is used to
join the two files based on a key field present in both the files. The input file can be separated by
white space or any delimiter.
Syntax:
$join [OPTION] FILE1 FILE2
TR
The tr command is a UNIX command-line utility for translating or deleting characters. It supports a
range of transformations including uppercase to lowercase, squeezing repeating characters, deleting
specific characters, and basic find and replace. It can be used with UNIX pipes to support more
complex translation. tr stands for translate.
Syntax :
$ tr [OPTION] SET1 [SET2]
UNIQ
The uniq command in Linux is a command-line utility that reports or filters out the repeated lines in
a file. In simple words, uniq is the tool that helps to detect the adjacent duplicate lines and also
deletes the duplicate lines. uniq filters out the adjacent matching lines from the input file(that is
required as an argument) and writes the filtered data to the output file.
Syntax :
//...syntax of uniq...//
$uniq [OPTION] [INPUT[OUTPUT]]
Shell Scripting – Decision Making
In programming, Decision making is one of the important concepts. The programmer provides
one or more conditions for the execution of a block of code. If the conditions are satisfied then those
block of codes only gets executed. Two types of decision-making statements are used within shell
scripting. They are –
If-else statement
case-sac statement / Switch statement
1. If-else statement
If else statement is a conditional statement. It can be used to execute two different codes based on
whether the given condition is satisfied or not. There are a couple of varieties present within the if-
else statement. They are –
if-fi
if-else-fi
if-elif-else-fi
nested if-else
PAGE 3
The syntax of if-else statement Example of if-fi
will be – Name="Satyajit"
if [ "$Name" = "Satyajit" ]; then
echo "His name is Satyajit. It is true."
if-fi
fi
if [ expression ]; then
Output
statements
His name is Satyajit. It is true.
fi
-v This prints out all the lines that do not matches the pattern
Print only the matched parts of a matching line, with each such part on a
-o separate output line.
PAGE 1
3.until statement
The until loop is executed as many times as the condition/command evaluates too false. The loop
terminates when the condition/command becomes true.
Syntax:
until <condition>
do
<command 1>
<command 2>
<etc>
done
Output:
$bash -f main.sh
0
1
2
3
4
5
6
7
8
9
10
PAGE 2