Linux administration

• UNIX is a multi-task, multi-user operating system
  • Developed from 1970s at Bell Lab, AT&T, USA
  • UNIX's main contributors: Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlRoy và Joe Ossanna
• Currently, UNIX is one of the most popular operating system in server market segment
• There are many UNIX-like operating systems of: Linux, BSD UNIX, HP-UX, AIX, Sun OS, Sun Solaris, Mac OS...

• shell is a command interpreter. It connects users and applications to the operating system
• shell is either CLI (command-line interface) or GUI (graphical user interface)
• In UNIX: sh is the first shell ever. sh uses CLI.
• In Linux, common CLI shells are bash, tcsh, ksh, zsh

• MULTICS (Multiplexed Information and Computing Service) is a research project to build a multi-task, multi-user started from 1965 (and ended in 2000) (http://multicians.org)
• Main use of MULTICS is in research
• In 1965, Ken Thompson, Dennis Ritchie, M. D. McIlroy, and J. F. Ossanna left MULTICS and decided to start a smaller scale project, initially named as UNICS
• UNICS = Uniplexed Information and Computing Service; the project renamed to UNIX
• In 1972 UNIX is re-writeen in C programming language

• PhD, Harvard University, 1968
• Known for: ALTRAN, B, BCPL, C, MULTICS, UNIX
• Notable awards: Turing Award (1983), National Medal of Technology (1998)...

• M.S. University of California, Berkeley, 1966
• Known for: UNIX, B, Belle, UTF-8, Golang
• Notable awards: Turing Award (1983), National Medal of Technology (1998)...

• PhD, Applied Mathematics, MIT, 1959
• Known for: Unix pipeline, component based software, spell, diff, join, graph, speak, tr

• Known for: UNIX, awk, AMPL, C programming language book
• Field of research: Computer science

• Occupation: Electrical engineer, programmer
• Known for: UNIX, nroff

• Linux is an UNIX-like, open source operating system:
  • The first version of Linux kernel released in Sep 17, 1991
  • Linus Torvarlds (Finland) is the author of the first Linux kernel
• A brief history:
  • 1983: Richard Stallman started GNU Project (GNU is Not UNIX) to create a system of free software that is compliance with UNIX
  • 1987: Prof. A. Tanenbaum created MINIX, a micro-kernel operating system to serve his university education and reseach purpose. Tanenbaum published all MINIX source code but MINIX is not free software until 2000.
  • 1991: Linus Torvarlds developed Linux using MINIX as a platform
  • Initially Linux uses many MINIX software packages, but later uses GNU packages including text editor emacs, software development GNU toolchain: gcc, autoconf, automake, libtool...

• Linus Torvarlds: kernel development
• Richard Stallman: GNU component development
• Developer community, enterprises etc develop 3rd party software
• GNU/Linux applications: supercomputer, server, desktop, smartphone, handheld devices, embedded devices, game device and some special-purpose devices such as home theater, digital security, system rescue...

• Nationality: Finland
• Education: M.S., University of Helsinki

• Initial: rms, website: https://www.stallman.org/
• Nationality: US
• Education: Harvard University (Bsc), MIT (MSc)
• Notable awards: MacArthur Fellowship, EFF Pioneer Award, ACM Software System Award

• Login prompt displayed
  • When Linux first loads after booting the computer
  • After another user has logged out
• Need to enter a username and password
• The login prompt may be graphical or simple text
• If text, logging in will present a shell
• If graphical, logging in will present a desktop
  • Some combination of mousing and keystrokes will make a terminal window appear
  • A shell runs in the terminal window

• The shell is where commands are invoked
• A command is typed at a shell prompt
  • Prompt usually ends in a dollar sign ($)
• After typing a command press Enter to invoke it
  • The shell will try to obey the command
  • Another prompt will appear
• Example:

$ date
Wed Jan  4 21:15:24 ICT 2017

• To exit from the shell, use the exit command
• Pressing Ctrl+D at the shell prompt will also quit the shell

• Quitting all programs should log you out

  • If in a text-only single-shell environment, exiting the shell should be sufficient
  • In a window environment, the window manager should have a log out command for this purpose

• After logging out, a new login prompt should be displayed

• Most commands take parameters
  • Some commands require them
  • Parameters are also known as arguments
  • For example, echo simply displays its arguments:

$ echo Hello there
Hello there

• Commands are case-sensitive
  • Usually lower-case

echo whisper
whisper
ECHO SHOUT
bash: ECHO: command not found

• Data can be stored in a file
• Each file has a filename
  • A label referring to a particular file
  • Permitted characters include letters, digits, hyphens (-), underscores (_), and dots (.)
  • Case-sensitive — NewsCrew.mov is a different file from NewScrew.mov
• The ls command lists the names of files

• There are many ways of creating a file
• One of the simplest is with the cat command:

$ cat > shopping_list
cucumber
bread
yoghurts
fish fingers

• Note the greater-than sign (>) - this is necessary to create the file
• The text typed is written to a file with the specified name
• Press Ctrl+D after a line-break to denote the end of the file
  • The next shell prompt is displayed
• ls demonstrates the existence of the new file

• There are many ways of viewing the contents of a file
• One of the simplest is with the cat command:

$ cat shopping_list
cucumber
bread
yoghurts
fish fingers

• Note that no greater-than sign is used
• The text in the file is displayed immediately:
  • Starting on the line after the command
  • Before the next shell prompt

• To delete a file, use the rm ('remove') command
• Simply pass the name of the file to be deleted as an argument:

$ rm shopping_list

• The file and its contents are removed
  • There is no recycle bin
  • There is no 'unrm' command
• The ls command can be used to confirm the deletion

• Typically, succesful commands do not give any output
• Messages are displayed in the case of errors
• The rm command is typical
  • If it manages to delete the specified file, it does so silently
  • There is no 'File shopping_list has been removed' message
  • But if the command fails for whatever reason, a message is displayed
• The silence can be be off-putting for beginners
• It is standard behaviour, and doesn't take long to get used to

• To copy the contents of a file into another file, use the cp command:

$ cp CV.pdf old-CV.pdf

• To rename a file use the mv ('move') command:

$ mv commitee_minutes.txt committee_minutes.txt

• Similar to using cp then rm
  • For both commands, the existing name is specified as the first argument and the new name as the second
• If a file with the new name already exists, it is overwritten

• The shell can making typing filenames easier
• Once an unambiguous prefix has been typed, pressing Tab will automatically 'type' the rest
• For example, after typing this:

$ rm sho

pressing Tab may turn it into this:

$ rm shopping_list

• This also works with command names
  • For example, da may be completed to date if no other commands start da

• Often it is desired to repeat a previously-executed command
• The shell keeps a command history for this purpose
  • Use the Up and Down cursor keys to scroll through the list of previous commands
  • Press Enter to execute the displayed command
• Commands can also be edited before being run
  • Particularly useful for fixing a typo in the previous command
  • The Left and Right cursor keys navigate across a command
  • Extra characters can be typed at any point
  • Backspace deletes characters to the left of the cursor
  • Del and Ctrl+D delete characters to the right

• a. Log in.
• b. Log out.
• c. Log in again. Open a terminal window, to start a shell.
• d. Exit from the shell; the terminal window will close.
• e. Start another shell. Enter each of the following commands in turn.
  • date
  • whoami
  • hostname
  • uname
  • uptime

• a. Use the ls command to see if you have any files.
• b. Create a new file using the cat command as follows:

$ cat > hello.txt
Hello world!
This is a text file.

Press Enter at the end of the last line, then Ctrl+D to denote the end of the file.

• c. Use ls again to verify that the new file exists.
• d. Display the contents of the file.
• e. Display the file again, but use the cursor keys to execute the same command again without having to retype it.

• a. Create a second file. Call it secret-of-the-universe, and put in whatever content you deem appropriate.
• b. Check its creation with ls.
• c. Display the contents of this file. Minimise the typing needed to do this:
  • Scroll back through the command history to the command you used to create the file.
  • Change that command to display secret-of-the-universe instead of creating it.

After each of the following steps, use ls and cat to verify what has happened.

• a. Copy secret-of-the-universe to a new file called answer.txt. Use Tab to avoid typing the existing file's name in full.
• b. Now copy hello.txt to answer.txt. What's happened now?
• c. Delete the original file, hello.txt.
• d. Rename answer.txt to message.
• e. Try asking rm to delete a file called missing. What happens?
• f. Try copying secret-of-the-universe again, but don't specify a filename to which to copy. What happens now?

• Make each program do one thing well. To do a new job, build a fresh rather than complicate old programs by adding new "features".
• Expect the output of every program to become the input to another, as yet unknown, program.
  • Don't clutter output with extraneous information.
  • Avoid stringently columnar or binary input formats.
  • Don't insist on interactive input.
• Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them.
• Use tools in preference to unskilled help to lighten a programming task

• Write programs that do one thing and do it well (DOTADIW)
• Write programs to work together
• Write programs to handle text streams, because that is a universal interface

• Files in UNIX/Linux are regular files, directories and devices
• By default, a UNIX/Linux program has
  • Standard input file stream (stdin): Use to read data from keyboard
  • Standard output file stream (stdout): Use to output data, often on screen
  • Standard error file stream (stderr): Use to output error messages,on screen
• Using I/O redirect we can change stdin, stdout, stderr of a program:
  • To change stdin: program < infile.txt
  • To change stdout: program > outfile.txt
  • To change stderr: program 2> errfile.txt

• Look for users those have user names begin with post:

grep ^post < /etc/passwd

## postgres:x:26:26:PostgreSQL Server:/var/lib/pgsql:/bin/bash

• Here < sign tells the grep program read data from /etc/passwd instead of read from stdin

• List files in the current directory and save output to a file:

ls > filelist
cat filelist | head -9

## 01-intro-backup.html
## 01-intro-backup.md
## 01-intro-backup.Rmd
## 01-intro.html
## 01-intro.md
## 01-intro.Rmd
## 02-gettingstarted.html
## 02-gettingstarted.md
## 02-gettingstarted.Rmd

• The > sign tells ls to output data to filelist instead of output to screen

• The file after > is overwritten
• If you do not want to overwrite, use >> to append output to the file

ls /etc/passwd /etc/xyz > out 2> err    # /etc/passwd exists, /etc/xyz does not exist

cat out

## /etc/passwd

cat err

## ls: cannot access '/etc/xyz': No such file or directory

• If you don't want error message to overwrite, use 2>> to append

• Redirect stdout và stderr to a file:

ls > filelist 2>&1 ## Redirect output to filelist, std error to output

or simpler:

ls >& filelist

• Pipe redirect output of a program to input of another program
• Examples:
  • ls lists all files in a directory
  • wc counts number of characters, words and lines
  • To count number of characters, words and lines in a directory, we use pipe (|) as follows:

ls | wc -l

## 28

# /etc/passwd exists, /etc/xyz does not exist
ls /etc/passwd /etc/xyz > out 2> /dev/null  # Don't want to see error messages
ls /etc/passwd /etc/xyz > /dev/null 2> err  # Only want to see error messages
ls /etc/passwd /etc/xyz >& /dev/null        # No output, no error message
grep abc < /dev/null                        # No input

cat err

## ls: cannot access '/etc/xyz': No such file or directory

cat out

## /etc/passwd

• 1.1. Use I/O redirect and pipe mechanism to redirect stderr of a program to stdin of another program
• 1.2. Use I/O redirect and pipe mechanism to redirect stdout and stderr of a program to stdin of another program
• 1.3. List all different shells of users in the systems, and calculate number of users using each shell, given that:
  • List of users is in /etc/passwd. The /etc/passwd file is a structured text file, where information of each user is in a line. Each line contains fields separated by :. List of fields is: user name, dummy password, user ID, group ID, user information in free text, user's home directory and user's shell
  • You can use UNIX/Linux utilities and using I/O redirect and/or pipe to solve this problem cut, sort, tee, uniq