OS Lab

Fundamentals and Automating

Week 4
01
FUNDAMENTALS
Must know command line skills and commands.
Files & Directories
Command: pwd
pwd stands for Print Working Directory. It prints the path of the working directory, starting from the root. pwd
is shell built-in command(pwd) or an actual binary(/bin/pwd). $PWD is an environment variable which stores
the path of the current directory.

The Tilde ( ~ )
The tilde (~) is a Linux "shortcut" to denote a user's home directory. Thus tilde slash (~/) is the beginning of a
path to a file or directory below the user's home directory.

Directory & Environment PATHs

There are two path concepts in Linux. Absolute path and relative path. The absolute path describes the
complete directory structure starting from the root (/). The relative path is based on the current directory.

The Environment path is a system variable that have a collection of absolute paths that allows the user to
call specific commands without giving the absolute or relative path for the command/program.
Files & Directories
Command: cd
cd command in linux known as change directory command. It is used to change current working directory.

Command: ls
ls command is a basic command in Linux used to List files and directories. ls command comes with so many
arguments and features

Command: touch
The touch command is a standard command used in UNIX/Linux operating system which is used to create,
change and modify timestamps of a file.

Command: cp
cp stands for copy. This command is used to copy files or group of files or directory. It creates an exact image
Files & Directories
Command: mv
mv stands for move. mv is used to move one or more files or directories from one place to another in file
system like UNIX.

Command: rm
rm stands for remove here. rm command is used to remove objects such as files, directories, symbolic links
and so on from the file system like UNIX. To be more precise, rm removes references to objects from the
filesystem, where those objects might have had multiple references (for example, a file with two different
names). By default, it does not remove directories.

This command normally works silently and you should be very careful while running rm command because
once you delete the files then you are not able to recover the contents of files and directories.
Files & Directories
Command: mkdir
mkdir command in Linux allows the user to create directories (also referred to as folders in some operating
systems ). This command can create multiple directories at once as well as set the permissions for the
directories. It is important to note that the user executing this command must have enough permissions to
create a directory in the parent directory, or he/she may receive a ‘permission denied’ error.

Command: rmdir
rmdir command is used to remove empty directories from the filesystem in Linux. The rmdir command
removes each and every directory specified in the command line only if these directories are empty. So if the
specified directory has some directories or files in it then this cannot be removed by rmdir command.

Command: alias
alias command instructs the shell to replace one string with another string while executing the commands.
When we often have to use a single big command multiple times, in those cases, we create something called
as alias for that command. Alias is like a shortcut command which will have same functionality as if we are
writing the whole command.
Files & Directories
Command: find
The find command in UNIX is a command line utility for walking a file hierarchy. It can be used to find files
and directories and perform subsequent operations on them. It supports searching by file, folder, name,
creation date, modification date, owner and permissions. By using the ‘-exec’ other UNIX commands can be
executed on files or folders found.

Command: locate
locate command in Linux is used to find the files by name. There is two most widely used file searching
utilities accessible to users are called find and locate. The locate utility works better and faster than find
command counterpart because instead of searching the file system when a file search is initiated, it would
look through a database. This database contains bits and parts of files and their corresponding paths on your
system. By default, locate command does not check whether the files found in the database still exist and it
never reports files created after the most recent update of the relevant database.
Managing Text Files
Command: cat
Cat(concatenate) command is very frequently used in Linux. It reads data from the file and gives their
content as output. It helps us to create, view, concatenate files. So let us see some frequently used cat
commands.

Command: less
Less command is linux utility which can be used to read contents of text file one page(one screen) per time.
It has faster access because if file is large, it don’t access complete file, but access it page by page.
less filename

Command: more
more command is used to view the text files in the command prompt, displaying one screen at a time in case
the file is large (For example log files). The more command also allows the user do scroll up and down through
the page.
Managing Text Files
Command: head
The head command, as the name implies, print the top N number of data of the given input. By default, it
prints the first 10 lines of the specified files. If more than one file name is provided then data from each file is
preceded by its file name.
head filename
Command: tail
The tail command, as the name implies, print the last N number of data of the given input. By default it prints
the last 10 lines of the specified files. If more than one file name is provided then data from each file is
precedes by its file name.
tail filename
Command: sort
SORT command is used to sort a file, arranging the records in a particular order. By default, the sort
command sorts file assuming the contents are ASCII. Using options in sort command, it can also be used to
sort numerically.
Managing Text Files
Command: grep
The grep filter searches a file for a particular pattern of characters, and displays all lines that contain that
pattern. The pattern that is searched in the file is referred to as the regular expression (grep stands for
globally search for regular expression and print out).
grep expression filepath
Command: diff
diff stands for difference. This command is used to display the differences in the files by comparing the files
line by line. Unlike its fellow members, cmp and comm, it tells us which lines in one file have to be changed to
make the two files identical.
The important thing to remember is that diff uses certain special symbols and instructions that are required to
make two files identical. It tells you the instructions on how to change the first file to make it match the
second file.
diff file1 file2

Command: wc
Managing Text Files
Command: cut
The cut command in UNIX is a command for cutting out the sections from each line of files and writing the
result to standard output. It can be used to cut parts of a line by byte position, character and field. Basically
the cut command slices a line and extracts the text. It is necessary to specify option with command otherwise
it gives error. If more than one file name is provided then data from each file is not precedes by its file name.

cut -b 3- filename from the third character in the line to the end of the line.

cut -b -3 filename from the first character in the line to the third character in the line.

cut -d "delimiter" -f (field number) file.txt to cut lines based on specific delimiter.
Help
Command: man
man command in Linux is used to display the user manual of any command that we can run on the terminal.
It provides a detailed view of the command which includes NAME, SYNOPSIS, DESCRIPTION, OPTIONS, EXIT
STATUS, RETURN VALUES, ERRORS, FILES, VERSIONS, EXAMPLES, AUTHORS and SEE ALSO.

man [COMMAND NAME]

Input/Output Redirection &
Piping
The work of any command is either taking input or gives an output or both. So, Linux has some command or
special character to redirect these input and output functionalities.

Types of Redirection

1. Overwrite
a. “>” standard output
b. “<” standard input
2. Appends
a. “>>” standard output
b. “<<" standard input

Examples

3. Insert text into file: echo “Hello World” > file.txt

4. Reading a file : cat < file.txt
Input/Output Redirection &
Piping
A pipe is a form of redirection (transfer of standard output to some other destination) that is used in Linux and
other Unix-like operating systems to send the output of one command/program/process to another
command/program/process for further processing. The Unix/Linux systems allow stdout of a command to be
connected to stdin of another command. You can make it do so by using the pipe character ‘|’.

Pipe is used to combine two or more commands, and in this, the output of one command acts as input to
another command, and this command’s output may act as input to the next command and so on.

Syntax:
command_1 | command_2 | command_3 | .... | command_N

Examples:

● Read file then sort it: cat file.txt | sort

● Print lines that contain specific word: cat file.txt | grep “word”
● View a file with the ability to scroll through it: cat file.txt | more
● Sort file and remove duplicate lines: sort file.txt | uniq
Questions?
02
SCHEDULE & AUTOMATE
TASKS
Automate and schedule services and tasks
Scheduling
Running one server, or a fleet of servers, involves some periodic cleanup tasks. Temporary files need deleting,
logs need rotating, caches need expiring, reports need running, and much more. You can either spend every
working day with an ever-growing checklist of things to do, or learn how Linux can schedule jobs to be run in
the wee hours when you’re sleeping.

Linux has two ways of scheduling jobs based on whether they’re regularly occurring. The cron system runs
jobs on a schedule that can be monthly, weekly, daily, hourly, or even per minute. The at system is used for
one-off jobs, such as to run a job over the weekend.

The Cron System

Cron is the main job scheduler in Linux. Named after the Greek word for time, chronos, cron runs a job for you
on a fixed schedule. Jobs can be anything that can be run on the command line including calling another shell
script.

People typically schedule intensive jobs at times when the system is expected to be underused such as
overnight. By separating the interactive user sessions from the heavy batch work, both classes get better
performance. Your Linux system almost certainly has a handful of jobs set to run overnight or at least on a
daily basis.
Scheduling
Configuring crontabs

The cron system has evolved over time. The utilities themselves have improved to allow more configuration
options. Distributions have also added more default settings to make common tasks easier.

The crontab, or cron table, is both a thing that stores a list of jobs for a user and the utility to manage the
table.

The simplest way to use crontab is to run it with the -e flag, which means you want to edit the current user’s
crontab. You are placed into an editor showing the current crontab. After you make changes and save, your
input is checked to make sure it’s a valid crontab. If the validation check fails, you are given an option to go
back so that you don’t lose your changes.

If you are root, the -u option allows you to supply a username so that you can view or edit that user. To edit
ali’s crontab, run crontab -e -u ali.

The -l flag displays the crontab rather than editing it.

Scheduling
Matching Times

The crontab itself has undergone some improvements in formatting, and most are built around the idea that
there are five columns to specify when the job is run and then the rest of the line contains the text to be
executed.

The columns, in order, are

1. Minute (0-59)
2. Hour in 24 hour time (0-23)
3. Day of month (1-31)
4. Month (1-12)
5. Day of week (0-7 with 0 and 7 being Sunday)

Each column must be filled in. If you want to match all values for a column, use an asterisk (*).
Scheduling
Some examples illustrate how the schedules work:

0 12 * * * - The minute is 0, the hour is 12, and it is to run on all months and days. This runs at 12:00 noon
every day.

0 0 1 1 * - The minute and hour are both 0 so, this means midnight. The day of the month and month are
both 1, which is January 1. This job runs at midnight on New Year’s day.

* * * * * - Runs every minute.

30 5 * * 1 - Runs at 5:30 a.m. every Monday.

All columns must match for the job to run. One interesting exception is that if you specify both the day of the
month and the day of the week (columns 3 and 5), the job will run with either match. Otherwise all columns
must match up for the job to run. So 30 5 1 * 1 runs at 5:30 on the first of the month and every Monday.
Scheduling
Spelling Out Month and Day Names

You can spell out the month and day of week names by using the first three letters. Not only is this easier to
remember, but it helps make more sense of the columns.

0 0 * jan sun runs at midnight on every Sunday in January. The whole file doesn’t have to be consistent, so
you can use numbers or names at your convenience.

Making Multiple Matches

The syntax you’ve seen so far does not allow for the same job to run at multiple times. If you wanted to run
the same job at midnight and noon, you would need two separate lines. Fortunately cron’s syntax has evolved
to allow this.

The first way to specify multiple times is to separate the items with a comma. For example, use the following
syntax to run a job at midnight and noon: 0 0,12 * * *

The first column is still 0 to indicate that the job runs when the minute is zero. The hour column is now 0,12,
which means the job will run when the hour is either 0 or 12, which is midnight and noon, respectively.
Scheduling
This also works when using the names of the months. A job can be run twice a day during June, July, and
August with this: 0 0,12 * jun,jul,aug *

The second way to run a job at multiple time periods is to give a range of values. Rather than jun,jul,aug, the
previous example could have been written with jun-aug. Or to run a job on the hour from 9:00 a.m. to 5:00
p.m: 0 9-17 * * *

This runs the first job at 9:00 a.m. and the last job at 5:00 p.m. Both these methods can be combined, such as
8-10,16-18.

Step Values

The next optimization is to provide an easy way to run a job by stepping over certain periods, such as to run a
job every 2 hours. You could do this with 0,2,4,6,8,10,12,14,16,18,20,22, but that’s messy!

Instead, run a job every 2 hours, on the hour with: 0 */2 * * *

Or perhaps every 30 minutes: */30 * * * *

Scheduling
Putting the crontab Together

So far you’ve looked at the first five columns, which collectively describe the time that the job will run. Any
remaining text on the line runs as a command at the appointed time. A line such as 0 0 * * *
/usr/local/bin/backup.sh

The command you run can be any valid shell code:

0 0 * * * if [[ ! -f /var/lock/maintenance ]]; then /usr/local/bin/backup.sh; fi

Nicknames

- @reboot—Run once after reboot.

- @yearly—Run once a year at midnight on January 1.
- @annually—Same as @yearly.
- @monthly—Run at midnight on the first of the month.
- @weekly—Run once a week on Sunday at midnight.
- @daily—Run once a day at midnight.
- @hourly—Run once an hour, on the hour.
Scheduling
Therefore, the following two crontabs are the same:

0 0 * * * /usr/local/bin/backup.sh
@daily /usr/local/bin/backup.sh

System crontabs

Some software packages need to bundle periodic tasks. For example, the sysstat package includes a valuable
tool called the system activity reporter, or sar. A cron job fires every 10 minutes to collect some stats, and
another job fires around midnight to archive the day’s statistics.

If the utility were to manipulate root’s crontab, it could accidentally get removed if the root user didn’t
understand why the entry was there. Removing the entry after the package is removed also becomes a
problem.

Therefore there is a second set of crontabs meant to be manipulated only by the root user. There is a shared
file called /etc/crontab and a directory containing individual tables in /etc/cron.d. The file is usually used by
the distribution itself to list any default jobs, or for the administrator to list any manually entered jobs. The
directory is most helpful for integrating with package management, where a package that needs its own cron
Scheduling
These system crontabs have one important difference: They contain a sixth column that goes in between the
schedule and the command to be run. This column indicates which user should be used to execute the job.

Example:
0 1 * * Sun root /usr/sbin/raid-check

Convenience crontabs

One problem with scheduling jobs by a specific time is that they all run at the same time. Often you don’t care
about the specific time a job runs, you just want it to run hourly, daily, weekly, or monthly. Cron is installed
with a configuration that has a directory for each of these time periods and will run the jobs in each directory
consecutively when the scheduled time comes up.

These directories are

- /etc/cron.hourly—Jobs here are run once an hour.

- /etc/cron.daily—Jobs here are run once a day.
- /etc/cron.weekly—Jobs here are run once a week.
- /etc/cron.monthly—Jobs here are run once a month.
Scheduling
The system guarantees that monthly jobs are run once every month, but you can’t say for sure that it’ll
happen exactly on the first of the month at midnight.

The files that go in these directories are just scripts. You do not include any schedule columns or user
columns. Most of these are placed there by installation scripts.
Questions?