This lab should be done with a partner of your choosing.
Lab 7 Goals:
- Implement a Unix shell program
- Practice creating and reaping processes:
fork, execvp, waitpid
- Practice using signals and writing signal handler code
- Practice executing commands in the foreground and background
- Practice implementing a queue to store the shell command history
- Gain yet more expertise in gdb and valgrind
Lab 7 Introduction
The setup procedure for this lab will be previous similar.
First, both you and your partner should run setup31 to
grab the starting point code for this assignment. Suppose users molly and tejas
which to work together. Molly (mdanner1) can start by running
[~]$ setup31 labs/07 tdanner1
Once the script finishes, Tejas (tdanner1) should run
[~]$ setup31 labs/07 mdanner1
For the next step only one partner should copy over the starting code
[~]$ cd ~/cs31/labs/07
[07]$ cp -r ~lammert/public/cs31/labs/07/* ./
[07]$ ls
Makefile cs31shell.c parsecmd.h parsecmd.o sleeper.c
Now push the changes to your partner
[07]$ git add *
[07]$ git commit -m "lab 7 start"
[07]$ git push
Your partner can now pull the changes.
In this case if Tejas wishes to get files Molly pushed, he would run
[~]$ cd ~/cs31/labs/07
[07]$ git pull
The starting point code includes:
Makefile,
cs31shell.c into which you should implement your shell,
and
sleeper.c which is a simple program that can be used to test your shell
in various ways (some described below). The starting point code also includes portions
of the parsecmd library, which will be used to parse command-line arguments given
by the user. The file
parsecmd.h declares the interface for the
command-line parsing library, particularly
parse_cmd, that you will use to
parse a string containing the user's command-line arguments into a list contained in argv.
The file
parsecmd.o contains the object code for the command-line parser itself.
Project Details
unix shell program
You use the unix shell all the time, but did you ever stop to think about what it
actually is? It turns out that the shell is actually just a program. That program does
the following things:
- prints a prompt and waits for user to type in a command line.
- reads in the command line entered by the user.
- parses the command line string into argv list.
- if it is not a built-in command, forks a child process to execute
the command, and waits for it to finish (unless the command in run
in the background, then the shell doesn't wait for the child to exit).
- else, if it is a built-in command, the shell program handles
the command itself (without forking a child process).
- repeat until the user enters the built-in command exit
to exit the shell program.
You will implement a Unix shell program that supports the following functionality:
- executes simple commands run in the foreground (e.g. ls -l)
- executes simple commands run in the background (e.g. ./sleeper &)
- implements the built-in command exit to terminate.
- implements the built-in command history that prints out
the N most recently entered command lines by the user (and each one's
command ID).
- implements running commands using !num syntax, where num
is the command with a matching command ID from the command history.
using the parsecmd library
In parsecmd.h is the function prototype for the parse_cmd function
that you can use this function to construct the argv array of
strings from the command line string. The argv array is passed into execvp.
The function comment describes how to call the function.
There are also constant definitions that you can use
in your shell (note: parsecmd.h is already #included at the top
of cs31shell.c, so your code can use anything it defines).
Note: The parse_cmd function is already implemented
for you. It is compiled into the parsecmd.o binary that is linked into
your cs31shell executable when you type make: you do not need
to implement the parse_cmd function, but just call it in your code
when you want to use it (much like you can call printf in your code
without implementing the printf function).
running a command in the foreground
When a command is run in the foreground, for example:
cs31shell> sleeper 2
your shell program should wait until the child process it forks exits.
To do this the parent should call waitpid passing in the
pid of the child process. See the Hints section below for more details.
running commands in the background
When a command is run in the background, for example:
cs31shell> sleeper 3 &
your shell program should NOT wait until the child process it forks exits, in this case.
Instead, after forking the child process,
it should go to step 1 (print out the prompt and read in the next command
line). The child process will execute the command concurrently with the
parent process handling other commands.
When a child run in the background exits, your shell program will need
to reap it by registering a signal handler on SIGCHLD. When the
parent shell process receives a SIGCHLD, the handler code will run
and it should call waitpid to reap the exited child (or children).
See the Hints section below for more details on signals.
Your shell should be able to run any number of processes in the background,
so if you type in quick succession:
cs31shell> sleeper &
cs31shell> sleeper &
cs31shell> sleeper &
cs31shell> ps
The ps program output should list all three sleeper child processes.
built-in commands
Shell built-in commands are executed by the shell process itself and
not by forking off a child process to execute them. Your shell
needs to support 3 built-in commands:
- exit: terminate the shell program (you can print out a goodbye
message if you'd like)
- history: list the 10 most recently entered command lines by the
user.
- Blank command lines (e.g., the user just entering Return)
should not be added to the history list. And, make sure your
shell doesn't crash when blank lines are entered.
- Command lines with invalid commands should be added
to the history list (e.g. if the user enters the command
abcd and there is no abcd executable file to run, the
abcd command line should still be added to the history list).
- !num (e.g. !5): executes command lines from the history list.
The command from the history list could be a run-in-the-foreground,
run-in-the-background, or a built-in command that your shell should
execute appropriately. The command line should be added into the history
list (i.e. executing !5 should not put !5 in the history list,
instead a copy of the command line associated with command ID 5
from the history list should be added to the history list).
See the Sample Output for some examples of history and !num commands.
history
The history command is a built-in command that prints
out the history list in order from first (oldest) to last
(most recently entered) command. For each element in the list, it
should print (1) its command ID and (2) its command line string.
See the sample output for an example of the history command.
The history list:
Your shell program should keep a list of the 10 most recently entered
command lines by the user. Use a constant definition for 10, but
try out some other values, like 20, 15 for MAXHIST.
For each command in the command history you should store:
- its unique command ID (an always increasing value):
1, 2, ..., 10, 11, 12, ... I suggest using an unsigned int, and
don't worry about overflow after running max unsigned int commands.
- its command line string (i.e. what the user entered at the prompt).
For example:
"ls -l -a"
The history list should be implemented as a fixed-size queue of history
structs: a circular array of
of MAXHIST buckets where new commands are added to one end and
old ones removed from the other end.
See the Thursday's lab fixed-size queue of ints example for more details on circular queues.
I suggest first implementing the Thursday lab circular queue of
ints program, and then once you get that working implement your
circular array of history structs in your shell program.
If you get stuck on the circular queue part, implement your
history list just as an array history list structs, where the first
history list element is always in bucket 0 and the last in
bucket MAXHIST-1, and then shift values over as you add new
command to the last bucket each time. Then, come back to this later
and try to turn this into it a circular queue (which is much more
efficient than having to shift values over to maintain the queue ordering).
!num
This is a built-in command for running a command from your history list.
This command should:
- Search your history list for a command with a matching command
ID. Remember that the command ID is not the position in
the history list, it is the unique number of the command in your
shell's execution history (i.e. !5 is the 5th command run by your shell,
!34 is the 34th command run by your shell).
- If a matching command ID is not found, then print out an error message.
- Otherwise, use the command line from the matching history command
ID, and re-execute it (this command now also becomes the most recent
command to add to your history list).
If the command from the history list is not a built-in command, then
your shell should run it just like it does any simple foreground or
background command (parse its command line into argv list, fork-execvp
and waitpid or not).
If the command from the history list is a built-in command (which could
only be the history command), then it should execute it directly just
like any built-in command.
See the sample output for examples of running all types of commands
that your shell needs to support.
Project Requirements
- Your shell should support running simple commands in the foreground
(e.g. ls -l)
- Your shell should support running simple commands in the background (e.g. ./sleeper &)
- Your shell should support the built-in command exit to
terminate.
- Your shell should support the built-in command history
that keeps a list of the MAXHIST most recently entered command lines
entered by the user. Use a constant definition for MAXHIST, and submit
your solution with it set to 10, but try your shell out with other sizes too.
- Your shell should support running commands from the history using !num syntax, where num is the command ID of a command from your
command history (e.g. !33 should execute the command with
command ID 33 from your history list). If a matching command num is not
found the current history list, then your shell should print out and
error message (command not found), otherwise
the command line from the matching command on the history list should
be executed (again).
- Use the execvp version of exec for this
assignment. See Hints section for examples.
- You need to add one signal handler on SIGCHLD signals for
reaping exited child processes that are run in the background.
- Use waitpid instead of wait.
- Whenever your code calls a function that returns a value, you
should have code that checks the return value and handles error
return values. You can call exit for unrecoverable errors,
but print out an error message first (printf or perror for system call
error return values).
- The only global variables allowed are those associated with
the history list and its state. All other program variables should be
declared locally and passed to functions that use them.
- Your solution should use good modular design and be well-commented.
The main function should not be much longer than that in the starting point
code. Think about breaking your program's functionality
into distinct functions.
- Your solution should be correct, robust, and free of valgrind errors
Example Output
Here is output from a run of my shell program:
sample output. Note when
my shell prompt is printed for jobs run in the background,
and how the history command and the !num built-in command works.
Useful C Functions and Hints
- Implement and test incrementally (and run valgrind as you go). Here
is one suggestion for an order to implement:
- Add a call to parse the input line into argv strings
- Add support for the built-in command exit
- Add support for running commands in the foreground (the parent
process, the shell, waits for the child pid that it forks off to exec
the command).
- Add support for running commands in the background (the parent
process, the shell, does NOT wait for the child pid that it forks off to run
the command). After forking off a child to run the command, the shell
program should go back to its main loop of printing out a prompt and
waiting for the user to enter the next command. You will need to add
a signal handler on SIGCHLD so that when the process that is running
in the background terminates, the shell reaps it. Use waitpid to
reap all child processes. Use the sleeper program to test:
cs31shell> sleeper &
cs31shell> sleeper 2 &
cs31shell> ps w
- Add support for the history list (implemented as a circular queue).
The operations on your circular queue are slightly different when the
queue is not yet full from when it is full and you need to replace the
oldest entry each time a new command is entered. Think about all state
you will need to keep to keep
track of the first element in the list (for printing out), the next
insertion spot, and the end of the list.
- Add support for !num built-in command that will run command
num from your history list. num is a command ID, which
is increasing as your shell runs commands. It is NOT the bucket index
into the history list.
- add debug printf stmts to help you see what your program is
doing (remember that the child process' address space is replaced
with the filename executable passed to execvp, so it will only
print debug code before its call to execvp). REMOVE all debug statements
from your submitted solution.
- You can make use of parsecmd library to get the argv list from the
command string in the history. If you have used good modular design
so far, you should be able to just make calls to existing functions to
execute the command. Read the parsecmd.h file comments to see how to
use the parse_cmd function.
- The maximum length of a command line is defined in parsecmd.h. You
can use the MAXLINE constant in your program.
- Some useful system calls: see the man page for each for
more information:
pid_t fork();
int execvp(const char *file, char *const argv[]);
// file: is the first string in the command line
// argv: is the whole command line list of strings (just like argv to main)
// (argv[0] is the same string as file, and you can pass
// argv[0] as the first argument value when you call this function)
pid_t waitpid(pid_t pid, int *status, int options);
// examples:
// wait for specific child process to exit (its pid is passed in pid param)
// pid = waitpid(pid, &status, 0);
// reap any child process that has exited (-1)
// but don't wait if none have exited (WNOHANG):
// pid = waitpid(-1, &status, WNOHANG);
sighandler_t signal(int signum, sighandler_t handler);
// example:
// signal(SIGCHLD, my_sig_child_hdlr);
// signal handler functions must have a prototype matching this:
// void my_sig_child_hdlr(int sig);
- Remember that you know how to define and use structs in C, and
that you know how to use strings in C. See past weekly lab code,
your lab solutions, and Prof. Newhall's C documentation.
Remember that you can compare individual char values directly, but you need
to use strcmp to compare two string values:
char str[20];
strcpy(str, "hello");
if(str[2] == 'x') {
printf("x in 2\n");
}
if(strcmp(str, "yo ho ho") == 0) {
printf("str seems like a pirate\n");
}
Remember if you dynamically allocate space for a string (using malloc),
you need to allocate a space at the end for the terminating null
character ('\0'), and that you need to explicitly free the space when
you are done using it (call free).
- Call fflush(stdout); after any calls to printf
to ensure the printf output is written immediately to the terminal
- Look at the weekly lab page and example code from lecture.
Also, look at the lecture slides on processes.
- When in doubt about what your shell should do, try running the
command in the bash shell and see what it does. For example,
here is a way to see how bash executes the history built-in command:
example of history and !num in bash
Cleaning-Up Zombie Children
Your correct shell program should reap all exited children and
not leave around zombie children. However,
in its development stages, you may find that it creates zombie processes
that it never reaps.
You should check for, and kill, all zombies after running your shell program.
To do this, after running your shell program, run ps in bash to see
if you have left any un-reaped zombie children (or any extra
instances of your shell program still running):
ps
233 pts/7 00:00:01 sleeper <zombie>
If so, send it (them) a kill signal to die using either kill -9 or pkill -9:
kill -9 233 # kills the process with pid 233
pkill -9 sleeper # kills all sleeper process
Submit
To submit your code, simply commit your changes locally
using git add and git commit. Then run git push while in the
labs/07 directory. Only one partner needs to run the final
push, but make sure both partners have pulled and merged
each others changes. See the section on using a shared repo
on the git help page.