UNIX Operating System

By Krishna

1
Objectives

• Introduction to OS concepts
• Introduction to Unix Operating System
• Unix System Architecture
• Introduction to Unix File System
• Starting to work with UNIX

2
• What is Unix?
• Need for Unix in today’s world?
• Evolution of Unix.
• Flavors of Unix.

What is Unix
• Operating System
– User Interface
– File Subsystem
– Process Subsystem
– Memory Management Subsystem
– I/O Subsystem
• Is a System software
• Can be defined as
– An Organized collection of software consisting of
procedures for operating a computer
• Provides an environment for execution of
programs
• Acts as an interface between the user and the
hardware of the computer system.
• Operating system interacts with the user in
two ways
– Operating system calls
Provides an interface to a running program
and the operating system. System calls in UNIX is
written in C.
– Operating system commands
Enables the user to interact directly
with the operating system.
• Evolution of Unix
• AT & T
– 1969 withdrew its team from MULTICS OS
• Ken Thompson and Dennis Ritchie developed
– File system
– Command Interpreter
– Set of utilities
• 1973 rewrote entire system in C.
• 1982 System III
• 1987 System V Release 3.0
• System V Release 3.2
• Offered free to academic and research inst.
• University of California ,Berkeley
– BSD (Berkeley Software Distribution)Unix
• First filled gaps left by AT&T
• Later rewrote as they wanted
– vi
– C shell
– Improved
» File system
» Mail system
» Symbolic links
– TCP/IP
– Offered free to many companies
Product Company Remarks

Xenix Microsoft Corporation Now discontinued

FreeBSD,Net Berkeley Software Commercial

BSD and Design venture started by
OpenBSD some of the
people who
created BSD
UNIX.Has a
version that runs
on PC
Solaris Sun Microsystems

AIX IBM

HP-UX Hewlett Packard

Digital UNIX DEC

Tru64 DEC

IRIX Silicon Graphics

SCO Open Santa Cruz Operation Replaced by SCO

Server UnixWare
SCO Santa Cruz Operation Runs on PC
UnixWare
Linux Red Hat,Suse,Caldera A very popular
and may other UNIX clone
Companies
• Standardization
• AT & T
– System V Interface Definition(SVID)
• X/Open(The Open Group) a consortium of vendors
and users
– X/Open Portability Guide(XPG)
• IEEE
– Portable Operating System Interface(POSIX)
• POSIX.1 C application program interface
• POSIX.2 shell and utilities
• 1998
– X/Open and IEEE joint initiative
• 2001
– Single UNIX Specification,Version 3(SUSV3)/IEEE
1003:1:2001(POSIX.1)
• 2002
– ISO approved above and called it ISO/IEC
9945:2002
LINUX and GNU
• Linus Torvalds
– Free UNIX
• Richard Stallman
– Free Software Foundation ,GNU
• LINUX is distributed under GNU General Public
License(source code public - mandatory)
Features Of Unix.
• The important characteristic features are
– Multiuser,Multitasking
– Portability.
– Security.
– Pipes.
– Redirection Tools.
– Software Development Tools.
– Stable and Reliable
– Shell programming.
– Hierarchical File System.
– Provides virtual memory support
– Devices are treated as files
– Supports the X-Window system
– Provides more than one type of shells (command
interpreters)
System Architecture.
• Major components of Unix are :
– Kernel
– Shell
– Utilities
– User Applications
• Unix System follows a layered Approach
• It has four layers
• The innermost layer is the hardware layer
• In the second layer, the kernel is placed.
• The utilities and the other application
programs form the third layer
• Fourth layer is the one with which the user
actually interacts.
Kernel
•
Kernel is that part of the OS which directly makes
interface with the hardware system.
•
Functions:
– Provides mechanism for creating and deleting processes
– Provides processor scheduling, memory and IO
management
– Does interprocess communication.

16
System Architecture.
User
User
User
User
SHELL
SHELL
UNIX OTHER
COMMANDS APPLICATIONS

HARDWARE

DATABASE KERNEL COMPILERS

PACKAGES

SHELL
SHELL

User User
User
User
Utility, command or shell script

shell
User Mode
Kernel
Mode
system calls
File I/O Memory Scheduler Interrupt
System System Manager and
Kernel
Exception
Buffer Handlers
Cache
How Unix Works.
LOGIN

SHELL ASKS FOR A COMMAND

USER TYPES COMMAND

SHELL EXECUTES
UTILITY TO CARRY
OUT COMMAND
USER INTERACTS WITH
UTILITY
SHELL PROMPTS FOR
NEXT COMMAND
USER TYPES CONTROL-D

LOGOUT
Unix User Account
• Your Unix user account will have a login name
and most probably a password.
• In addition, a number of other attributes are
associated with a user account in Unix. The
four most important attributes are:
– Home Directory
– Login Shell
– User ID
– Group ID
Login Process
• Once you log in, Unix creates a process for you, and
runs the login shell as a program in that process.

• For the login shell process, Unix makes your home

directory as the current directory.

• Therefore, after login, by default, you will be working

with your home directory.
• When you log into Unix, your login shell also
executes a startup script for you from your
home directory.
• The name of this script depends upon the login
shell. For Bourne shell, this file is “.profile” file.
For C-shell, this file is “.cshrc” file.
• To customize your login environment, you can
modify these startup scripts.
Using Unix Commands

• Types of commands
1) Built-in commands :
These commands are part of the shell

2) External commands
These commands are separate programs or
files that are stored under a filesystem
Unix Command Structure

Unix Command line structure.

command [options] [arguments]
The Shell
•
A Utility program that comes with the unix system.
•
Features of Shell are:
– Interactive Processing
– Background Processing
– I/O Redirection
– Pipes
– Shell Scripts
– Shell Variables
– Programming Constructs

25
Process management
•
A process is a program in execution
•
Several processes can be executed simultaneously in a
UNIX system.
•
A process is generally created using the “fork” system call.
•
The process that invoked the “fork” system call is the
parent process and the newly created processed are called
as child processes
•
Fork create a child process

26
CPU Scheduling
•
Unix uses “Round-Robin” scheduling to support its
multi-user and time-sharing feature.
•
Round-Robin fashion of scheduling is considered to
be oldest,simplest and widely used algorithm.
•
Here, every process is given a time slice(10-100
millisec.)

27
File management
•
Unix uses a hierarchical file system with “/” as its
root.
•
Every non-leaf node of the tree is called as a
directory file.
•
Every leaf node can either be a file or an empty
directory

28
File system

dev bin tmp home etc var lib usr

spool
sh inittab bin src
console ls password
user1
lp0
user2

29
• Exploring the file system.
A file system is the collection of files and the hierarchy of
directories on a system. The time has now come to
escort you around the file system.
First, change to the root directory (cd /), and then enter ls -
F to display a listing of its contents. You'll probably see
the following directories : bin, dev, etc, home, install, lib,
mnt, proc, root, tmp, user, usr, and var.
Now, let's take a look at each of these directories.
• /bin
/bin is short for “binaries”, or executables, where may essential system
programs reside. User ls –F /bin to list the files here. If you look down the list you
may see a few commands that you recognize such as cp, ls , and mv. These are
the actual programs for these commands. When you use the cp command , for
example, you’re running the program /bin/cp.
• /dev
/dev the “files” in /dev are device files they access system devices and
resources like disk drives, modems, and memory. Just as your system can read
data from a file, it can also read input from the mouse by accessing /dev/mouse.
filenames that begin with fd are floppy disk devices. Fdo is the first floppy disk
drive, and fd1 is the second. You may have noticed that there are more floppy
disk devices than the two listed above: these represent specific types of floppy
disk. For example, fd1H1440 accesses high density, 3.5 disketters in drive1.
The following is a list of some of the most commonly used device files. Even though
you may not have some of the physical devies listed below, chances are that
you’ll have drivers in /dev for them anyway.
• /dev/console refers to the system’s console that is, the monitor connected
directly to yur system.
• The various /dev/ttys and /dev/cua devices are used for accessing serial ports.
/dev/ttys0 refers to “COM1” under MS-Dos. The /dev/cua devices are “callout”
devices, and used with a modem.
• Devices names beginning with hd access hard drives. /dev/had refers to the
whole first hard disk, while /dev/hda1 refers to the first partition on /dev/had.
• Device names that begin with sd are SCSI drives. If you have a SCSI hard drive,
instead of accessing it throuth /dev/had , you would access /dev/sda.SCSI tapes
are accessed via st devices, and SCSI CD-ROM via sr devices.
• Device names that begin with lp access parallel ports. /dev/1p0 is the same as
“LPT1” in the MS-DOS world.
/use/bin
/usr/bin is the real warehouse for software on any Linux system, containing most of
the executables for programs not found in other places, like /bin
• /user/etc
Just as /etc contains essential miscellaneous system programs and
configuration files, /usr/etc contains miscellaneous utilities and
files, that in general, are not essential to the system.
/user/include
/usr/include contains “include files” for the C compiler. These files
(most of which end in .h, for “header”) declare data structure
names subroutines, and constants used when writing programs in
C. files in /usr/include/sys are generally used when programming
on the UNIX system level. If you are familiar with the C
programming language, here you’ll find header files like “stdio.h”
which declare function like printf().
• /usr/lib
/usr/lib contains the “stub” and “static” library equivalents for the
files found in /lib. When compiling a program, the program is
“linked “ with the libraries found in /usr/lib, which then directs
the program to look-in /lib when it needs the actual code in the
library. In addition, various other programs store configuration
files in /usr/lib.
• /usr/local
/usr/local is much like /usr it contains various programs and files not
essential to the system, but which make the system fun and
exciting. In general, programs in /usr/local are specialized for
your system consequently, /usr/local differs greatly between
Linux system.
UNIX classification

• The Owner
• Group member
• other users

35
UNIX COMMANDS

36
Printing present working directory
• In Unix user’s location in the file system
hierarchy is known as “current working
directory”.
• When the user logs in, the system places the
user automatically in the user’s “home
directory”.
• The syntax:
• Example :
$pwd
/home/guest/project
37
Printing date and time
• date displays the current data and time. A superuser
can set the date and time.
• Syntax
– date [options] [+format]
• Common Options
– -u use Universal Time (or Greenwich Mean Time)
– +format specify the output format
– %a weekday abbreviation, Sun to Sat
– %hm onth abbreviation, Jan to Dec
– %y last 2 digits of year, 00 to 99
– %D MM/DD/YY date
– %H hour, 00 to 23
– %M minute, 00 to 59
– %S second, 00 to 59
– %T HH:MM:SS time 38
• Examples:

$date
Mon Jun 10 09:01:05 EDT 2006

$date -u
Mon Jun 10 13:01:33 GMT 2006

$date +%a%t%D
Mon 06/10/06

$date '+%y:%j'
96:162

$date +%d ->> day of the month 1 to 31

22

$date +%m ->> Month of the year in numeric value

09

$date +%D ->> full date format mm/dd/yyy

$date “+%H %M %S” ->> Hour, Minute and Second use double quotation for space

$date +%y ->> year (yy)

39
$ date +%a  short name for week

$ date +%A -> Full name for week

$ date +%b abbreviated Month Nme (jan)

$ date +%B full name for month (January)

$ date +%j  gives day of the year

$ date +%u –> gives number of week (1 to 7)

$date +%U gives number of week in a year (1 to 52)

who - list current users
• who reports who is logged in at the present.
• Syntax
– who [am i]
• Examples
$who
rashmi ttyp1 Apr 21 20:15 (apple.acs.ohio-s)
rangitha ttyp2 Apr 21 23:21 (worf.acs.ohio-st)
radika ttyp3 Apr 21 23:22 (127.99.25.8)
ravi ttyp4 Apr 21 22:27 (slip1-61.acs.ohi)

41
sudhir ttyp5 Apr 21 23:07 (picard.acs.ohio-)
leela ttyp6 Apr 21 23:00 (ts31-4.homenet.o)
prakash ttyp7 Apr 21 23:24 (data.acs.ohio-st)

• Who am I

$who am i
• Guest ttypc Apr 21 23:38 (mars)

42
Displaying calendar
• cal displays calendar.
• Syntax
– Cal [-smjy13] [[month] year]
• Examples
• Options
– -1 display single month output
– -3 display prev/current/next month output
– s display Sunday as first day of the week
– m display Monday as first day of the week
– y display calendar for the current year
• $cal
-- displays current months calendar
43
echo a statement
• The echo command is used to repeat, or echo, the argument you give it
back to the standard output device.
• Syntax
– echo [string]
• Common Options
– \n new-line (SVR4)
Examples
$ echo Hello Class or echo "Hello Class"

– To prevent the line feed:

– $ echo -n Hello Class or echo "Hello Class \c”

44
Format and print data
• Printf – format and print data
• Syntax:
– Printf FORMAT [ARGUMENT]…
Options:
print arguments according to Format
\” double quote
\\ backslash
\a alert
\b backspace
\n new line
\r carriage return
\t horizontal tab
\v vertical tab
%% a single % 45
Changing the passwords
• Passwd - update a user’s authentications
• syntax
– Passwd options [username]
Options:
-d This is a quick way to disable a passwd
-x This will set the maximum password life time, in days.
-w This will set the number of days in advance user will
begin receiving warnigs that his passwd will expire.

46
Printing name of machine
• hostname (uname -n on SysV) reports the
host name of the machine the user is logged
into
• e.g.:
– $hostname
mars

47
Print system information
• uname has additional options to print information
about system hardware type and software version.
• Option:
– a option prints all the information.
– s kernel name
– r kernel release
– v kernel version
– m machine
– p processor
– i hardware platform
• E.g:
– $uname –a
Linux mars 2.4.20-8 #1 Thu Mar 13 17:54:28 EST 2003 i686
i686 i386 GNU/Linux
48
Files & Directories

49
Objectives
• File Related Commands

• File Permissions

• Directory Related Commands

50
File related commands
FILE OPERATIONS COMMANDS
Listing contents of the ls
directory
Copying a file cp

Moving a file mv

Removing a file rm

Displaying a file cat

51
Listing the directory contents
ls
Syntax :ls [options] [file….]
options: -l list in long format
-a list all file including those
beginning with a dot
-i list inode no of file in first column
-s reports disk blocks occupied by file
-R recursively list all sub directories
-F mark type of each file
-C display files in columns
52
Listing the directory contents
$ ls –l
groupid
total 6 Link counter Size in bytes
-rwxr-xr-x 1 user1 training 12373 Dec 15 14:45 a.out
drwxr-xr-x 2 user1 training 4096 Dec 22 14:00 awkpro
-rw-r--r-- 1 user1 training 12831 Dec 12 13:59 c
File name
-rw------- 1 user1 training 61440 Dec 15 11:16 core
-rw-r--r-- 1 user1 training 255 Dec 20 14:29 cs

userid

Date &Time of
File access Modificaion
permission
Type
of file

53
Command grouping

• wc one.txt ; ls –l one.txt
• (wc one.txt ; ls –l one.txt) > two.txt

54
Command - cp
Used to copy files across directories
Syntax
cp <source file> <new file name>

Example
cp file1 file2

55
Command - cp
• -p preserve following file attributes
– owner id
– Group id
– permissions
– Last modification time

56
Command - mv
Used to move a file or rename a file

Preserves the following details

– owner id
– group id
– permissions
– Last modification time

-f suppresses all prompting

-i prompts before overwriting destination file

57
Command - rm
Used to remove a file
– Syntax : rm file(s)
-f suppresses all prompting
-i prompts before overwriting destination file
-r will recursively remove the file from a directory (can
be used to delete a directory along with the content )

Caution: Use “i” option along with “r” to get notified

on deletion

58
• More paging output
If a file is too large for its contents to fit in one screen, it will scroll off your screen when
you view it with cat. Press ctrl+s to stop it.
$ more <file name>

• File: know the file types

There are basically three types of files in UNIX (ordinary,directory and device), you may
often require to know more about these files. For instance , a regular file may contain
plain text, a C program or executable code etc.
$ file <filename>

Cmp: Comparing files.

If you require to know whether two files are identical so that one of them can be deleted.
There are three comands in the UNIX system that can tell you that. In this section,
we’ll have a look at the cmp command.
$ cmp <filename1> <filename2>
Note: the above two files are compared byte by byte
Comm: finding what is common
Comm requires 2 sorted files, and compares each line of the first file with its
corresponding line in the second.
$ comm <sort1> <sort2>
File access permissions
• Refers to the permissions associated with a file
with respect to the following

• Permission Levels
– User (owner) (u)
– Group ( staff etc.) (g)
– World (guest, anonymous and all other users) (o)

• Permission Settings
– Read (r)
– Write (w)
– Execute (x)
60
File access permissions
• No read permission does not allow the user to:
– List the contents of directory
– Remove the directory
• No Write permission does not allow the user to :
– copy files to the directory
– remove files from the directory
– rename files in the directory
– make a subdirectory
– remove a subdirectory from the directory
– move files to and from the directory

61
File access permissions

• No execute permission does not allow the user to:

– display the contents of a directory file from within the

directory

– change to the directory

– display a file in the directory

– copy a file to or from the directory

62
Changing permissions -
• chmod u+x file_name chmod
Syntax:
chmod <category> <operation> <permission>
<filename(s)>
Or
chmod <octal number> filename

Octal Number
r -4
w -2
x -1
$ chmod 744 xyz
this will set rwx for user, r for group, r for others
63
$ ls –l # use to list the seven attributes of all files in lthe current
directory.

• Ownership : When you create a file you automatically become

its owner. The third column shows ‘kumar’ as the owner of all
these files.

• Group Ownership: When a user account is opened by the

system administrator he simultaneously assigns the user to
some group

Note: UNIX filenames can be very long (up to 255 char)

Here -d option listing directory attributes

$ ls –ld <dir Name>

not list all sub
Note : if you are simply using <ls –d> will
directory in the current directory.
U user ‘+’ assign permission
G group ‘-’ remove permission
O othere ‘=‘ assign absolute permission
A all
$ chmod u +x <filename>
$ chmod ugo +x <filename>
Absolute assignment: if assigns only those permission
that are specified along with it , and removes other
permission
$ chmod a=r <filename> # only read permission allowed
to the file.
Directory creation
Command Syntax
mkdir [OPTION] DIRECTORY

$mkdir <path>/<directory>
$mkdir –m <directory>
$mkdir –p <directory1>/<directory2>/<directory3>

66
Directory creation

• Example:
– mkdir project1
• This creates a directory project1 under current
directory

• Note:
Write and execute permissions are needed for
the user to create a directory
67
Directory removal

rmdir command removes directory

Syntax
– rmdir <directory name>
Example
Removes project1 directory in the current directory
– rmdir project1

68
Directory removal
Remove multiple directories
rmdir pos1 pos2
Remove the directory recursively
rmdir –p dir1/dir2/dir2

Rule: rmdir can be executed to a directory if it is

empty and not the current directory

69
Editors

70
vi editor
Vi is a visual editor used to enter and edit text files.
– A screen-oriented text editor
– Included with most UNIX system distributions
– Command driven
Categories of commands include
– General administration
– Cursor movement
– Insert text
– Delete text
– Paste text
– Modify text

71
Editing commands
The vi editor is invoked by the following command
– vi <filename>

• Text insertion / replacement

– i - inserts text to the left of the cursor

– a - inserts text to the right of the cursor
– I - inserts text at the beginning of the line
– A - appends text at end of the line
– o - opens line below
– O - opens line above
– R - replaces text from cursor to right
– s - replaces a single character with any
number of characters
–S - replaces entire line
72
Editing commands

• Deletion
– -x - to delete character at cursor position
– -3x - to delete 3 characters at cursor position
– dw - to delete word
– -2dw - to delete 2 word
– dd - to delete a line
– 2dd - to delete 2 lines

73
Editing commands
• Yanking
–Y - copy line into buffer
– 3Y - copy 3 lines into buffer
–p - copy buffer below cursor
–P - copy buffer above cursor
• Save and quit
– :w - to save
– :w! - to name a file (:w! filename -> save as)
– :x - save and quit
– :q - cancel changes
– :q! - cancel and quit
74
Search & replace
commands
:s/thee/the change the first occurrence of
thee with ‘the’.

To change every occurrence of a character string between

two lines,

:#,#s/old/new/g where #,# are replaced with

the numbers of the two lines.
:%s/old/new/g to change every occurrence in
the whole file.
75
Customizing vi
• :set number
• :set nonumber

76
Navigation

Backspace Space

h j k l

the quick brown fox the quick brown fox

w w w $

the quick brown fox the quick brown fox

2 w ^
the quick brown fox

b b b

77
Unix Utilities to manage
ordinary files

78
cat
cat is concatenate files and print on the standard
output
cat command takes the input from the keyboard and
send the output to the monitor
We can redirect the input and output using the
redirection operators
– $ cat>file1
– Type the content here
– press <ctrl d>
– $ cat file1
– Displays the content of the file
– $cat>>file1
– Will append to the content of the file 79
General purpose utilities

wc
– A filter used to count the number of lines,words and
characters in a disk file or in the standard input.
– -l - displays the number of lines
– -w - displays the number of words
– -c - displays the number of characters

80
General purpose utilities
• find The find command will recursively search the indicated directory tree to
find files matching a type or pattern you specify.

Syntax
find directory [search options] [actions]

Common Options
– For the time search options the notation in days, n is:
+n more than n days
n exactly n days
-n less than n days

– time that the file was last accessed or changed

-atime n access time, true if accessed n days ago
-ctime n change time, true if the files status was changed n days ago
-mtime n modified time, true if the files data was modified n days ago

81
-name filename true if the file is named filename.
-print print the pathnames found (default for SVR4, not for BSD)
• Examples
– 14 -rw-r--r-- 1 frank staff 6682 Feb 5 10:04 library
– 6 -r--r----- 1 frank staff 3034 Mar 16 1995 netfile
– 34 -rw-r--r-- 1 frank staff 17345 Feb 5 10:04 standard
– 2 -rwxr-xr-x 1 frank staff 386 Apr 26 09:51 tr25*

82
• To find all files newer than the file, library:
$ find . -newer file4 -print
./tr25
./standard

• To find all files with .dat extention:

find . –name “*.sh” –print
• We could search for any file name containing the string "ar"
with:
$find . -name \*st\* -ls
326584 7 -rw-r----- 1 frank staff 6682 Feb 5 10:04 ./library
326585 17 -rw-r----- 1 frank staff 17351 Feb 5 10:04 ./standard

• -mtime and –atime :

Use to find unaccessed or unmodified for the time
Find . –mtime -1 –print

83
• -size Locating large files
Files above 2Kb
$ Find . –size +2

• The operators –and and –or

$ find . \( -name “*.sh” –or -name “*.txt” \)
$ find . -name "*.txt" -and -size -1

• The Action on the selected files –exec and –ok with find command
$ find . -atime -2
Note: -ok will give a confirmation before removing a file

• Find the directory or sub directory and files (-type)

$ find . –type –d ak
$ find . –type –f add.sh
I/O redirection
< file redirect standard input from file
> file redirect standard output to file
2> fileredirect standard error to file
2>&1-merge standard error with standard output
$ cat > abc
$ ls –l > outfile
$ cat xyz abc > outfile 2> errfile
$ cat xyz abc > outfile 2>&1

85
Meta characters

* 0 or more characters
? 1 character
[ - ] matches any one character between the brackets
[^ ] not matches any one character in the brackets
^[ ] ---starting with
[ ] $ --ending with

86
Simple Filters
Filters are programs that takes its input from the std i/p file,
processes it and sends it to the std o/p file.
Commonly used filter commands
– Sort Ordering a File
– Grep Seraching For a Pattern
– Cut Slitting a File Vertically
– Head Displaying the Beginning of a File
– Tail Displaying the End of a File
– Paste Pasting Files
– Tr Translating Characters
– Nl Line Numbering
– Uniq Locating Repeated Lines

87
sort
Sorts the contents of the given file based on the first char of each line.
• Syntax
sort [options] [+pos1 [ -pos2 ]] file
Common Options
• -k start key at POS1
• -f fold lower case to upper case
• -n numeric sort at last
• -o outfile output file
• -r reverse the sort
• -t char use char as the field separator character
• -u for unique names in a file
• -c Checks if file is sorted
• -o Accept output file name as an argument.
• -ln gives the line number 88
• Example:
To sort file named “name”
$sort file

To sort file named “name” using 2 field

$sort –k2 name

To sort in reverse order using 2 field

$sort -r –k2 name

To sort numerically using 2 field

$sort –nk2 name
89
• Sort the first field with pipe delimiter
$ sort –t “|” –k1 <filename>
$ sort –t “|” +1 <filename>

Here argument +1 indicate that sorting start after skipping the first field. To sort on
the third field you should use.
$sort –t “|” +2 <filename>

• To check whether the file has actually been sorted.

$sort –c <filename>
$sort –t “|” +3 –c <filename>

• Note: Sort is one of those few UNIX filters that also accept the output filenames
as an argument . This is done with the –o option.
$ sort –o <output filename> -k2 <filename>

$sort <filename> | uniq –d

• The –d (duplicate) option selects only one copy of the repeated lines or
intersection
$sort <filename> |uniq –u
The –u (unique)option selects only the non-repeated lines (union)
grep
Grep -Global Regular Expression Printer
Is used for searching regular expressions or patter search.
Syntax
– grep <options> <pattern> <filename(s)>
– Common Options
– -i ignore case
– -c report only a count of the number of lines containing
matches.
– -n display the line number along with the line on which a
match was found
– ^[] beginning with
– [^ ] Not beginning with
– [ ]$ ending with
– [ ^ ]$ Not ending with
– ‘ ^ ‘ beginning with SPACE
91
• Examples
• In the first we’ll search for the number 15:
• $ grep '15' num.list

• to count the number of lines matching the search criterion:

$ grep -c '15' num.list

• search for all lines that begin with a space:

$ grep '^ ' num.list

• all lines that don’t begin with a space:

$ grep '^[^ ]' num.list

• Here we search for all lines that begin with the characters 1
through 9:
$ grep '^[1-9]' num.list
92
patterns
* - matches 0 or more characters
• [^pqr] - Matches a single character which is not p ,q or r
$ grep ‘^[ ^ pqr ]’ <filename>

• ^pqr -Matches pqr at the beginning of the line

$ grep ‘^[ pqr ]’ <filename>

• pqr$ -Matches pqr at the end of the file

$grep ‘[ pqr ]$’ <filename>

• pqr$ -Not Matches pqr at the end of the file

$grep ‘[ ^ pqr ]$’ <filename>

“.” - Matches any one character

\ - ignores the special meaning.
– Grep “New\[abc\]” filename

93
• To display all the pattern beginning with a.
$ grep “a.*” <filename>
$grep “[aA]g[ar][ar]wal” <filename>

Displaying line number

$grep –n naresh <filename>

• To search for patter which has space between

words
$grep “naresh kumar” <filename>
• Head
• By default it will show 10 lines of a file
$ head <filename>

• Displays the top n lines of the file

Can specify top n lines to be displayed
$ head -3 file1

95
Filter command - tail
• Displays the end of the file
Can specify last n lines to be displayed
$ tail -3 file1

• Can also specify the line number from which the data
has to be displayed
$ tail +5 file1
• Last three line in a file
$ tail -3 file1

96
Filter command - tr
• tr - Transtlate filter used to translate a given set of characters
usage :
– tr [ a-z ] [ A-Z ] < <filename>
• option -s can be used to squeeze the repeated characters.
• Translate ‘n’ with ‘m’
$ tr ‘n’ ‘m’ < <filename>

• Translate |/’ with ‘~’

$ tr ‘|’ ‘~’ < <filename>

• Translate upper case to lower case

$ tr ‘[ a-z ]’ ‘[ A-Z ]’ < <filename>

• Translate SPACE to ‘|’

$tr ‘ ‘ ‘|’ < <filename>

97
Command piping
• Allows the output (only the standard output) of a
command to be sent as input to another command.
• Multiple pipes may appear in one command line.
Example:

$ cat * | wc
$ cat fil1 | head | wc -l

98
Filter command – tee
• tee command allows the normal output to the standard
output, as well as to a file
• Useful to capture intermediate output of a long
command pipeline for further processing or debugging
purpose.
• The use of the “tree” facility that saves the output in to
the file and also display it on the terminals.
Example
– who | tee userlist
– cat - | tee file1 | wc -l

99
Filter command – cut
Used to extract specified columns from the o/p of certain
commands
-c used to extract characters
-d Delimiter for fields
-f Field no.
Examples
$ cut –c 1-5 <filename> or $cut –c 5 <filename>
Assume that we have filename sample.txt with following values.
Welcome to world of computing
Welcome to ibm
Thanks
Now the above example cut the first five character from 1 -5 and
display on the terminal.

100
$ cut -c2-5 file1
In the above example it will cut from 2 char to n-1 char (51=4) and display
on the terminal.

$ cut -d “|” -f2,3 file1

Here will look into the pipe delimiter and cut from field 2 and field 3
$ cut -d “|” -f2,3 file1 |cut –c 1-5 |tr “[ a-z ]” “[ A-Z ]”

Logical Line Numbe (nl)

$ nl <filename>

$ nl –w2 <filename>
Which increase the width
$ nl –s “>” <filename>
Which gives delimiter seperators.
Filter command – paste
Paste is used to fix two cut portions of the file
vertically

-s -->Pastes the contents of file2 below file1

-d -->Specify delimiter

$ paste -d”|” file1 file2

$ cut –d “|” –f2 <filename> |tee <filename2>

$ paste –d \| <filename2> <filename3>

102
Processes
A process is an instance of a running program PID (process Identifier)

$ echo $$
659 /* the process no of the current shell*/

The PID of your login shell doen’t change as long as you are logged in when you log
out and log in again, your login shell will be assigned a different PID

Parent & Children

just a file has a parent, every process also has one. This parent itself is another
process, and a process born from it is said to be its child.
Example : $ cat <fileName>

From the above example, a process representing the “cat” command, is started by
“sh” process (the shell) this cat process remains active as long as the command
active. “sh” is said to be parent of ‘cat’

103
• Built in command of the shell like pwd,cd etc
doesn't create any process.
Processes
Process related commands

– ps
– kill
– wait
– sleep

105
• Process - a program in execution

• When program is executed , a new process is created

• The process is alive till the execution of the program
is complete

• Each process is identified by a number called pid.

106
Login shell

As soon as the user logs in, a process is created which

executes the login shell.

Login shell is set for each login in /etc/passwd

directory.

107
The ps command is used to display the characteristics of a
process or attributes of a process.
$ ps
$ ps –l
$ ps –f
It fetches the pid, tty, time and the command which has started
the process.
– -f lists the pid of the parent process also.
– -u lists the processes of a given user
– -a lists the processes of all the users
– -e lists all the processes including the system processes

108
• Display process of a USER
$ ps –f –u
$ ps –a
Process creation
There are three distinct phases in the creation of a process (fork, exec,
wait)
Fork : When you enter a command at the prompt the shell first create a
copy of itself, when a process is forked in this way the child gets a new
PID.
Note: The forking is responsibility of multiplition of process in the system.\

• EXEC: The parent then overwrites the images that is has just created
with the copy of the program that has to be executed.
• WAIT: The parent then executes the wait system call
the keep waiting for child process to complete when
a child process has completed execution. It sends a
termination signal to the parent. The parent is free to
continue with its other function.
Background process

Enables the user to do more than one task at a time.

If the command terminates with a ‘&’ UNIX executes the command at the
background

110
Controlling background processes
jobs
– List the background process
fg %
– Runs a process in the foreground
bg %
– Runs a process in the background
• Running the job in Background
The ‘&’ is the shells operator used to run a process in the
background ( i.e not wait for its death)
$ sort –o <filename1> <filename2> &

111
• nohup: logout safely (nohangup) background jobs to
run, even though when a user logout
Note: ‘&’ should be used with ‘nohup’
$ nohup sort <filename> &

kill command
• kill : Kills or terminates a process
• Kill command send a signal to the process
– The default signal is 15 ( SIGTERM)
• Kill -9 (SIGKILL)
– Terminates the process abruptly

112
• Killing the last background job
<$!>: stores the PID of the last background job
$ sort –o <filename> <filename>
345
$ kill $!

Note: Some programs simply ignore the interruption, and continue

execution normally in that case, the process can be killed with
signal no ‘9’
$ kill -9 121
$ kill –n 9 121 <<sco UNIX>>

Note: You can kill all processes in your own system except the login
shell, by using a special argument ‘0’
$ kill -9 $$ << $$ stores pid of current shell >>
$ kill -9 0 << same as above >>

• Nice: job execution with low priority unix offers the ‘nice’ command, which is used with
‘&’ operator to reduce the priority of jobs
$ nice wc –l <filename>
$ nice wc –l <filename> &
Note: Nice values range from 0 to 39
$ nice –n 15 wc –l <filename> &
On a Linux machine, there are hundred's of processes, that are continuously
running for some or the other tasks. Linux Kernel does a fantastic job in
mediating between these processes and allotting CPU to these processes.

• AT & BATCH : executable later UNIX provides very sophisticated facilities to

schedule a job to be run at a specified time of a day.
$ at 1:29am #asking for the path name
$ /root/naresh/<filename>
$at 15
$at pm
$ at noon
$ at –l (list of job schedule)
Remove schedule jobs
$ at –r 39894383
• With the help of Nice command in Linux you can set process
priority. If you give a process a higher priority, then Kernel will
allocate more cpu time to that process.
• By default when a programme is launched in Linux, it gets
launched with the priority of '0‘.
• The column that starts with "NI" shows the nice value(priority of
the process).
• nice -10 <command name> Will set a process with the priority of
"10".
• Running nice command with no options will give you the priority
with which the current shell is running.
• A major misconception about nice command is that nice -10
<command name> will run that process with -10(a higher priority).
• In order to assign -10 priority to a command then you should run it
as shown below.
• .
• nice --10 <command name>
• Make a note of the double --, the first one is the command
option(-), and the second one is the numerical -(minus).
• Note: You can only use nice command to change the process
priority when you launch it. You cannot use nice command to
change the priority of an already running process.
• Some examples of renice command is as follows:
• renice -4 -p 3423 (this will set the priority of process id no 3423
to -4, which will inturn increase its priority over others)
• renice 13 -p 3564 -u Admin (this will set the priority of the
process id 3564 to 13, and all the process owned by user
"sarath" to the priority of 13)
• renice 14 -u sarath,satish -g custom (this will set all process
owned by "sarath","satish" and also the group "custom" to 14)
tar
•How to Compress and Extract Files Using the tar Command.
•The tar command on Linux is often used to create .tar.gz or .tgz archive files,
also called “tarballs.” This command has a large number of options, but you
just need to remember a few letters to quickly create archives with tar. The
tar command can extract the resulting archives, too.
•Compress an Entire Directory or a Single File
•tar –czvf naresh.tar directoryname
•tar –czvf naresh.tar directoryname
•tar -cvzf python.tar python

•Here’s what those switches actually mean:

•-c: Create an archive.
•-z: Compress the archive with gzip.
•-v: Display progress in the terminal while creating the archive, also known as
“verbose” mode. The v is always optional in these commands, but it’s helpful.
•-f: Allows you to specify the filename of the archive.
Untar
•How to Compress and Extract Files Using the tar Command.
•Exclude Directories and Files
In some cases, you may wish to compress an entire directory, but not
include certain files and directories. You can do so by appending an --
exclude switch for each directory or file you want to exclude.
tar -czvf ak.tar ak1 --exclude=*.ksh

Extract an Archive
It’s the same as the archive creation command we used above, except the -
x switch replaces the -c switch. This specifies you want to extract an archive
instead of create one.
tar -xzvf ak.tar
“gzip" Command
•The "gzip" command is a common way of compressing files within Linux and
therefore it is worth knowing how to compress files using this tool.
•gzip filename
•How To Decompress A File Using The "gzip" Command
•If you have a file that has already been compress you can use the following
command to decompress it.
•gzip -d filename.gz
•Force A File To Be Compressed
•To force the "gzip" command to do its stuff simply run the following command:
•gzip -f filename
•How To Keep The Uncompressed File
•By default when you compress a file using the "gzip" command you end up
with a new file with the extension ".gz".
•gzip -k filename
•Get Some Stats About How Much Space You Saved
•gzip -l filename.gz
•The information returned by the above command is as follows:
•Compressed size ,Uncompressed size ,Ratio as a percentage &Uncompressed filename
•Valid: gzip -t test.sh.gz ->no out put if valid.
“Bzip2" Command
•1. Compress a single file
This will compress file.txt and create file.txt.bz2, note that this will remove
the original file.txt file.
bzip2 file.txt
•2. Compress multiple files at once
This will compress all files specified in the command, note again that this will
remove the original files specified by turning file1.txt, file2.txt and file3.txt
into file1.txt.bz2, file2.txt.bz2 and file3.txt.bz2
bzip2 file1.txt fil
•Compress a single file and keep the original
You can instead keep the original file and create a compressed copy.
bzip2 -c file.txt > file.txt.bze2.txt file3.txt
•Decompress a bzip2 compressed file
To reverse the compression process and get the original file back that you
have compressed, you can use the bzip2 command itself or bunzip2 which is
also part of the bzip2 package.
•bzip2 -d file.txt.bz2 OR
•bunzip2 file.txt.bz2
“grep, egrep,fgrep and rgrep" Command
•grep -inh --colour=auto ak *.*
Here grep option -h allows us to suppress to display file name and with -n
option grep numbers lines within a file. --colour=auto highlights match
egrep
egrep is extended version of grep or egrep is equal to grep -E. Egrep supports
more regular expression patterns. Lets search for lines which contains exactly
two consecutive p characters:
egrep p{2} *.* sort1
grep -E p{2} *.* sort1
egrep -C 0 '(a|r)e' sort1
Fgrep or the Fixed grep or grep -F
Fgrep searches for complete string and doesn’t even recognize special
characters as part of regular expression even if escaped or not escaped
fgrep -C 0 '(ar)e' sort1
fgrep -C 0 '(a|r)e' sort1
Rgrep is same as grep -r. It is useful in doing recursive grep search. It searches
the whole directories and its sub-directories for the pattern
rgrep, unlike grep and egrep, can recursively descend directories. The
traditional way of performing this kind of search on UNIX systems utilizes
the find command in conjunction with grep. Using rgrep results in much better
performance. See also xargs command.
THE ENVIRONMENT
• History: it will show the history of the prompt.
$ history
$ history -5
$ history -5 10 # lists between 5 & 10
Repeating previous Command
$ r #korn
$ !! # bash
• With the event number as argument
$ r 38 # korn
$ !38
• Executing Previous Commands by context:
$ r v # repeat last command beginning with ‘v’
$ ! V # bash
• Preventing Accidental overwriting of a file:
The ‘noclobber’ option prevents you from accidentally overwriting the
contents of a file ,you can invoke that feature by using.
$ set –o noclobber #no more overwritting files with
$ set +o noclobber #enable
 File already exists message in korn shell

 Cannot overwrite existing file in bash shell

$ set –o ignore eof # <ctr –d> won’t log you out

 The ENV variable (ksh)

The korn and bourne shell use two startup files on logging. The
first is the user’s own (.profile) second file that contains most
of the korn shell settings.
$ echo $env
/etc/passwd file

• Sample entries

root:x:0:0:root:/root:/bin/bash
xfs:x:43:43:X Font Server:/etc/X11/fs:/bin/false
suresh:x:500:500:SURESH:/home/suresh:/bin/bash
perumal:x:501:501::/home/perumal:/bin/bash
user:x:502:502::/home/user:/bin/bash
user1:x:503:503::/home/user1:/bin/bash

124
Shell Programming

125
Flavors of shell
• Bourne shell sh developed by Stephen Bourne
• C shell csh ,, ,, Bill Joy
• Korn shell ksh ,, ,, David korn
• Bourne again bash Shell distributed with linux
shell

126
Shell features
• Displays the shell prompt and reads the command typed by the
user.

• Interprets the command and classifies it as internal (built-in) or

external command.

• If it is a NOT a built-in command, searches for the command in

the PATH specified directories and executes that command if it is
found.

127
Using the test command

• To compare two integers using test following

operators are available:

-eq (equal to) =

-ne (not equal to) <>
-lt (less than) <
-le (less than or equal to)<=
-gt (greater than)
-ge (greater than or equal to)

128
Using the test command

• Please note that you can only compare two

integers using the test command.

• The syntax for this comparison is:

test integer1 operator integer2

OR
[ integer1 operator integer2 ]

129
Using the test command

• To compare two strings using the test command,

following operators are available:

– string1 = string2 (equal to, please note it is a single =)

– string1 != string2 (not equal to)
– string1 (string is not NULL)
– -n string1 (string is not NULL and exists)
– -z string1 (string is NULL and exists)

130
Using the test command

• The syntax for this string comparison is:

test string1 operator string2

OR
[ string1 operator string2 ]
OR
test operator string
OR
[ operator string ]

131
Using the test command

• To check a file type/access permissions using the test

command, following operators are available:

– -s file (file is not empty)

– -f file (file is not a directory)
– -d file (file is a directory)
– -r file (file is readable)
– -w file (file is write-able)
– -x file (file is executable)

132
Thank You

133