Introduction

Command shells are a powerful way of interacting with a computer. However, they offer little in the way of comfort. It's no wonder that the two big desktop systems, Mac OS and Windows, moved most users away from the shell to easier-to-learn window-icon-mouse-pointer environments. But shells are having a resurgence. Mac OS X contains a fully-functional shell, Microsoft is shipping their own shell with new server OSes. Linux has always had the shell, and for a long time you couldn't effectively use a Linux machine without it. Efforts to make Linux more friendly have widely succeeded, but the downside is that many new users don't understand the shell.

This guide is intended to give you a gentle introduction to life at the command prompt. It can't cover everything. Shells have been in use for more than a quarter century and were designed by some smart folks. They're powerful tools, and you can't learn it all at once. But a little learning goes a long way!

There are many more encyclopedic shell guides out there. Here's a list of a few:

• Bash by example -- Good examples of typical stumbling blocks.
• Advanced Bash-Scripting Guide -- A textbook on learning bash. Probably the best source if you really want to learn bash. Despite it's title, it's for beginners.
• Bash Guide for Beginners -- The opposite of the above. It is an advanced guide with lots of detail. It's difficult for beginners to get much from this book, but it has a lot to offer experts.

I swear the "Advanced Guide" and "For Beginners" aren't jokes! They really should swap names. Unlike those references, this guide is intended to touch a range of skills briefly. It's designed to be consumed in one sitting.

In examples (gray background), I'll write

$ echo "hello world"
hello world

where "$" is the prompt (the thing before the cursor on each new command line). Lines without starting $'s are the output of previous commands. Comments on commands appear in red, and keystrokes appear in parenthesis, like (tab).

A lot of ideas used in shells are programming ideas. A big idea is the shell variable . You can set a shell variable by

$ export name="value"

and you can see the value of a variable by

$ echo $name

The "$" dereferences the variable name. "echo" is just a command that prints things.

Some variables are set for you every time you open a shell. These are often called environment variables. Let's look at some important ones:

$ echo $SHELL
/bin/bash
$ echo $HOME
/home/grad/mgratton
$ echo $PATH
/usr/lib/qt-3.3/bin:/usr/kerberos/bin:/usr/local/bin :/usr/bin:/bin:/usr/bin:/usr/games:/usr/share/namd-2.6 :/usr/NX/bin:/home/grad/mgratton/.local/bin

What about the other variables we looked at? Well, $HOME is occasionally useful for telling programs where your home is. It's a lot faster than typing "/uhome/grad/reallylongusername/". Alternatively, you could change your name, but that seems like a bother. $PATH is a very intimidating character at first, but it is really just a list of directories separated by ":".

When you try to run a command without specifying which directory it's in, the shell helpfully searches for the command you might mean. It looks for the command in each directory in $PATH, in order. As an example,

$ fortune
We'll cross that bridge when we come back to it later.
$ which fortune
/usr/bin/fortune
$ /bin/fortune
bash: /bin/fortune: No such file or directory
$ ./fortune
bash: ./fortune: No such file or directory

fortune probably gave you some other amusing quote. That's what it does. "which" asks the shell in which of the $PATH directories does the command "fortune" appear. It only appears in "/usr/bin", so we can be sure that's the executable that's running when we type "fortune". Now we can try to ask to run a different fortune, like the one in /bin or the one in this directory (./), but fortune isn't there, so it doesn't work.

Let's set up our own spot for executables.

$ mkdir -p ~/.local/bin/
$ cp ~mgratton/public_html/shell/cowsay ~/.local/bin/ $ chmod u+x ~/.local/bin/ $ fortune | ~/.local/bin/cowsay

If you're not on a department machine, replace the second line with

$ wget www.math.duke.edu/~mgratton/shell/cowsay $ mv cowsay ~/.local/bin

Okay, what's with all of the new commands? mkdir makes a new directory. The '-p' option creates any parent directories that don't exist. "~" is shorthand for $HOME. It's very useful. Then we copy the file 'cowsay' to the new directory. Finally, chmod makes cowsay executable. Cowsay provides the vital function of printing messages inside the speech bubble of an ascii-art cow (it's by Tony Monroe and used here for educational purposes only, though it is Free Software). Notice the strange way we called it -- more on that later. It is a pain to always type "~/.local/bin" in front of our commands! Let's fix this by changing the $PATH:

$ export PATH="$HOME/.local/bin/:$PATH"

The variables on the right side of the "=" will be dereferenced before PATH is set, so effectively, we've just prepended another directory. Okay, time to try it out.

$ fortune | cowsay

It's never been easier to talk to a cow.

The bad news is that the $PATH will revert to normal after we close this shell. To make this permanent, we need to add it to our .bashrc file. "RC" files are configuration files in UNIX, and the "." just makes it invisible. Mostly, these just live in your home, "~". So let's edit ours.

$ pico ~/.bashrc

Scroll to the bottom and add

export PATH="$HOME/.local/bin:$PATH"

Use control-x (^X) to exit and save.

All the commands in your .bashrc are executed in order whenever a shell starts, so it's a good place to customize your shell environment. This section looks at three little things to make your shell use more comfortable. As you add to your .bashrc, you'll need to do

$source ~/.bashrc

each time so that bash re-reads the contents of the file and your changes take effect.

Prompts: It's nice to have some extra information at the prompt. To change the prompt, we just change the shell variable $PS1. There are a lot of special "escape sequences" that tell bash to fill-in parts of the prompt with system data. If you see a prompt you like, add the whole export PS1=blah line to your .bashrc.

$ export PS1="\u@\H$"
mgratton@bender$ username and computer name
$ export PS1="\u@\H:\w$"
mgratton@bender:~$ u@c:current directory
$ export PS1="\[\033[1;34m\]\u\[\033[0m\]@\[\033[0;32m\]\h\[\033[0m\]:\w\$ "
mgratton@bender:~$ as above, but with swank colors

I recommend the second choice. It's nice to see which computer you're on, which directory you're in, and to remind interlopers that this is your session. More information on building your own custom prompt can be found at this page.

Aliases: You can also change the default behavior of commands (and define some new commands) through aliases. Here's an example:

alias ls="ls -BF --si"
alias la="ls -A"
alias ll="ls -l"

The first redefines the "ls" list command to always include some handy options (B: don't show backup files, F: add decorations like / to directory names). The next two define new commands "list all" and "long list" that show all of the files in a directory (including hidden files and backups) or show the file list in gory detail (with permissions, file size, and more). Another handy alias for many folks is

alias mat="matlab -nojvm -nosplash"

which runs matlab in the shell without fancy graphics.

A complete .bashrc: Here's a complete .bashrc that you may copy. It's the same as the above, plus a jot of code to speed things up.

if [ "$PS1" ]; then
    alias ls="ls -BF --si"
    alias la="ls -A"
    alias ll="ls -l"
    alias l="ls -CF" # yes, I'm that lasy
    alias mat="matlab -nojvm -nosplash"
    # set a fancy prompt
    PS1="\u@\H:\w$"
    # If this is an xterm set the title to user@host:dir
    case $TERM in
    xterm*)
        PROMPT_COMMAND='echo -ne "\033]0;${USER}@${HOSTNAME}: ${DIRSTACK}\007"'
        ;;
    *)
        PROMPT_COMMAND=''
        ;;
    esac
fi

export PATH=".:$HOME/.local/bin:${PATH}"
export PRINTER=lw4d

Up until now, everything here has been pretty sedate and, frankly, boring. Now it's time to do some cool stuff.

Three special characters allow you to move inputs and outputs of programs around. They are the pipe "|", the output redirect ">" and the input redirect "<". We've already seen the pipe in action:

$ fortune | cowsay

Pipes send the output (stdout) of the program on the left into the input (stdin) of the program on the right. So how can we make the cow say something else? How about

$ echo "hello" | cowsay
$ echo $PATH | cowsay
$ pwd | cowsay

Here's a more practical example. This converts a pdf to a ps, but instead of saving the file, it's just shipped off to the printer.

$ pdftops test.pdf - | lpr

the "-" tells pdftops to send its output to stdout. Lots of programs follow this convention.

Output redirection lets you take messages that would be printed to the console and put them somewhere else, like a file. Here's a silly example:

$ fortune | cowsay > cow

Now try "pico cow", and you'll see our four-footed friend. More useful uses include saving the errors from a compiler to a file so you can read and search it later.

More advanced output redirects.

$ echo "hello" > cow
$ echo "world" >> cow

The double sign, ">>" appends the output to the file rather than blowing away the file and putting the new output there. It's also occasionally handy to redirect stderr (where error messages are printed to). You might do

$ noisy_cmd >& logfile

or

$ noisy_cmd >& /dev/null

The first will take all the noise from noisy_cmd and dump it in a log file. The second takes all of the output and writes it to a special file -- the dreaded /dev/null. /dev/null isn't really a file, its just a black hole where data goes to die.

Input redirection lets you take the contents of a file and have it fed into a program that's expecting you to type in entries at a prompt. If you have a program that reads from stdin (scanf in C, or cin in C++) then

$ prog < input

will supply the contents of input to the program "prog".

Another main function of a shell is to control the programs you're running. The most common job control is something like

$ firefox &

The "&" tells bash to run the program, but return the shell to you. If you close the terminal (thereby killing the shell), firefox will also go away. Now, what if you forget the "&"? Try this:

$ firefox
(^Z)
[1]+ Stopped firefox
$ bg
[1]+ firefox &
$

Press control-Z and you have your prompt back. Bash reminds you that job 1 (called firefox) has been stopped (suspended, really). You can "background" the job with "bg". As you might have guessed, there's a whole list of job control functions. The most basic are

• ^z -- suspend
• ^c -- kill
• bg -- background suspended job(s)
• fg -- foreground a suspended job
• jobs -- list running jobs

The real power of bash is in the programming-type features: if statements and loops. You can use these from the prompt, or put them in files called shell scripts. Here's a basic loop operation (don't do this unless you want all of the jpg's in this directory to be resized!):

$ for fname in *.jpg; do convert -resize 320x240 $fname; done

This takes all files ending in ".jpg" and runs convert on them to resize them. Notice

• for [varname] in [list of strings] -- supply the variable name you want. You can put any list of strings you like. Wildcards like the one I used are really common.
• semicolons -- these mark the end of lines. You could just hit enter instead and type on the next line. The effect is the same.
• do ... done --- Between do and done, you can put as many commands as you like. $fname is changed to the next string in the sequence on each run.

Try this one out:

$ for fname in ~/*; do du -sh $fname; done

This will print the size of each file or directory in your home. du is "disk usage", and -sh makes it print sizes in KB, MB, or GB chunks, adding up all of the files in a directory.

It takes a long time to add up all of the files in a directory. What if we want to skip directories? That's a job for if:

$ for fname in ~/*
> do
> if [ -f $fname ]; then
> du -sh $fname
> fi
> done

Here, instead of ";", I've just hit return between statements. Bash prompts me with a ">" to continue until the for loop is closed by "done."

The if command has a similar structure to for:

• if [ test ] -- WARNING you MUST put a space just after then "[". It's actually a program called "test" being called in short hand.
• then ... fi -- just like do ... done from before. These enclose the payload of the if.

We used the "-f" test, which checks if the argument is a regular file. You can read-up on all of the many, many tests by doing

$man test

Another common operation is to chop a filename into parts, say dividing "stem.end" into "stem" and "end". For this, bash has some basic string processing tools.

$ MYVAR="hello.jpg"
$ echo ${MYVAR%.*}
hello
$ echo ${MYVAR##*.}
jpg

The ${variable...} syntax is the same as the $variable syntax, except it lets us add extra bits. % searches for the shortest match to our query (a "." followed by anything) starting at the back of the string. ## searches for the longest match to the query from the front of the string. You can guess how "#" and "%%" behave, or you can see the easy-to-read guide. That's a great place to learn more about shell programming.

Bash also has a lot of keyboard commands to make typing easier. The most necessary is tab. Pressing tab on a partially typed file or command name will fill it in with the full name provided you've typed enough for it to be unique. If you haven't, hit tab again to see a list of all partial matches. Try this:

$ head ~/.local/bin/cow   (press tab)
$ head ~/.local/bin/cowsay ##
## Cowsay 3.03
##
## This file is part of cowsay. (c) 1999-2000 Tony Monroe.
...

head prints the first ten lines of a file. In this case you can see who wrote cowsay. To see all the programs in your $PATH that start with "pdf", try

$ pdf   (press tab twice)
pdf2djvu   pdfcrop   pdfjadetex   pdftex   pdftoppm
...
$ pdf

Bash returns the cursor to the end of the command, and now we can choose the name we want from the list. You can really speed up your shell usage by hitting tab whenever you have enough of a file or directory name to be unique. For "~/.local/bin/cowsay", I type "~/.loc(tab)/b(tab)/cows(tab)" to get what I want.

Bash uses a system called "readline" to allow you to edit each line. This system has a set of standard commands that you can use. A good page of these is here. In brief:

• ^w erase word
• ^u erase from here to start of line
• ^a move to start of line (leaving all the text alone)
• ^e move to end of line

There are tons more, and you can redefine these to use other keystrokes, but that's beyond our scope. See "man bash" for more info.

Here's a list of commonly used commands that are "shell friendly", that is, they read and write to stdout and work in the UNIX way. They read strings separated by whitespace and return ... strings separated by whitespace. You can do tons of stuff with these.

Help:

• [man] "manual" get help on a command. Usually quite technical.
• [info] Like man, but designed to be more in-depth and friendly.

Looking at a file:

• [cat] "concatenate" Reads several (at least one) files, concatenates their contents, and prints it to stdout. Handy for printing a short file.
• [head] First 10 lines of a file
• [more] Reads a file if you do "more file" or from stdin and gives you a page of the file at a time.
• [less] "less is more" Works like less, but you can search, go backwards, and do other things.
• [tail] First 10 lines of a file

Moving files around:

• [ls] "list" See the manual for ALL the options :)
• [cp] "copy" Nice options are "-r" for recursive to copy directories, and "-i" for interactive to ask before it clobbers any files.
• [mv] "move" Useful for renaming! Also has a "-i" option.
• [rm] "remove" Has a "-r" to delete directories and their contents, and a "-i" for safety.
• [ln] "link" Creates a link, like an alias. Try "ln -s filename linkname". The "-s" is for symlink, which is almost always what you want.
• [tar] "tape archive" Makes compressed archives of files. The most common usages are "tar -czvf filename.tar.gz directory/" and "tar -xzvf filename.tar.gz"
• [chmod] "change modifier" Change the permissions of a file.
• [touch] Change the last access time of a file to "now". If the file doesn't exist, create it.

Big powerful commands:

• [grep] "gnu regular expressions" Searches for a regular expression (special type of pattern matching) in a file.
• [sed] "stream editor" Powerful find/replace tool, among other things.
• [find] Searches for a file. Use it like "find ./ -iname *.tex" to find all tex files in this directory.

Manipulate a text file:

• [cut] Removes part of each line in a file. Useful for columns of data.
• [paste] Puts extra text on each line of a file. Useful for concatenating column-wise.
• [sort] You guessed it!

Network operations:

• [ssh] "secure shell" Run it like ssh username@host. You can omit the username, and it'll fill in your own.
• [scp] "secure copy" Run it like scp username@host:path/to/filename ./. Copies remote files here, or vice versa.
• [wget] "web get" Download a file or a whole website.

Ole' fashioned commands:

• [users] Everyone logged onto this machine.
• [finger] Prints some information about a user.

Fun:

• [fortune] :)
• [yes] Try it. Press ^c, then try "yes no".
• [seq] Print a sequence of numbers. You can turn the for loop into a real C-like for loop with this. Also try "seq 1 12", then sing that song from Seasame Street.

---

By Michael Gratton (04/17/2008) mgratton@math.duke.edu