Basic Unix Command-Line

Yesterday I got my hands on some Unix tutorials on IBM developer works website. I must say this is a very good site for tutorials for the stuffs they put in there are always well described. I have been working on a Linux box for about a year and a half and I merely know much about it. Other than the few commands that I need for daily usage, I don’t know much else about it. So I started studying from Part 1 of the tutorial series. Here is a short note of what I found -

Unix’s power is in its command line. If you know it well, you can perform your tasks much faster and with greater control. Lets take a quick look at one of the powerful features: command chaining. We use pipe (|) in between two commands to combine them. The purpose of the pipe (|) is chaining. In a command, you can chain processes together using pipes, sending the data of one command to the next command. For example, to find the list of unique filenames in the folder hierarchy rooted at the current working directory, you can type the following at your shell prompt:

find . -type f -print | sort | uniq

This command line combines three separate utilities: find, sort and uniq.

Input and Output to/from a command:

find always takes the contents of the file system as its input data. However, both sort and uniq require data entry or input from the standard input device (stdin). Most often, you provide stdin using the keyboard.

A typical UNIX command-line utility reads from stdin and writes to stdout. Some utilities read data from system resources and write results to standard output device (stdout).

In addition to using stdin and stdout, UNIX commands can emit error messages to a special outlet for diagnostics. The outlet is called the standard error device (usually referred to as stderr). You can use variants of the redirection syntax to do just that: |& (pipe), >& (create) and >>& (append) stdout and stderr simultaneously. Redirection is explained just next.

Redirect:

Changing the source and destination of a process’s data is referred to as redirection. You redirect stdin to read data from a file or other source; and you can redirect stdout and stderr (separately) to write data somewhere other than the terminal window. In many cases, as in the original find command shown earlier, you can also redirect utilities to consume and produce data from and for other utilities.

You can use > to save the output of the entire sequence to a file (destroying the existing contents of the file, if any). You can also use >> to append the results to an existing file. For example, the following command redirects the output of grep command to a file.

grep ‘error’ server.log > result.txt

Here we have used grep is used to search for lines containing the given input. In the above example it outputs all the lines in server.log file that contains the string ‘error’. Note that, the result.txt file will be replaced once this command is run. If we want to append the result of this command to the file, we can use >>.

Another helpful redirection is <. It is used to read from a file and feed the output to some command. That means we are redirecting stdin to read from a file.

Now lets look at a few examples that will utilize command chaining and redirection.

1. Create a copy of any directory, including symbolic links, with tar:
   
   tar cf – /path/to/original | \
   (mkdir -p /path/to/copy; cd /path/to/copy; tar xvf -)

   The first tar archives the directory /path/to/original and emits the archive file to stdout; the hyphen (-) used with the create (c) option specifies stdout. The command in parentheses is a subshell. mkdir -p creates the named directory, including any intermediate directories that need to be created; and cd changes to the new directory. The second tar reads an archive from stdin and expands it in place; the hyphen used with the extract (x) option refers to stdin.

2. xargs reads data from a pipe and runs a specified command for every line read. This is a very useful command, helpfull when you want to perform some command on each line of a command’s output. For instance, if you want to find all Web pages on your server that reference www.example.com, you can use this command line:
   
   find / -name ‘*html’ -print \
   | xargs grep -l ‘www.example.com’ \
   | less -Opages

   Here xargs consumes the filenames from find and runs grep -l repeatedly to process every file, no matter how many files are named. (grep -l prints the name of the file if a match is found and then stops further matching in that file.) less allows you to page through the results and saves the list in the file named pages. The result is a list of filenames that contain the string “www.example.com”.

Thats it for today then. What I have tried here is to put down my findings from the mentioned article into my own way so that I can later quickly recall the stuffs I have learnt. Do let me know if you find it useful and how I can make it better.