Due Dates
-
Part1, checkpoint: before 11:59pm, Wednesday Sept. 27 push to git repo your solution to Part 1 (should be in the
part1/
subdirectory.You may not use late days on lab checkpoints, but you should still complete Part1 if you do not complete it by the checkpoint due date.
-
Part2: before 11:59pm, Wednesday Oct. 4 push to git repo and your solution to Part 2 (should be in the
part2/
subdirectory.
Lab2 Partners
This lab will be done with your Lab 2 Partner
See the documentation on the course webpage about working with partners. It is a brief guide for effective practices and my expectations for how CS students should be working together on labs.
Lab 2 Goals
-
Learn socket programming by implementing two versions of a TCP socket client-server program.
-
Implement a specific message protocol, and test correctness by seeing that your client/server can communicate with other group’s server/client.
-
Gain more practice with pthreads multi-threaded programming and synchronization.
-
More practice with C programming, C strings in particular.
Overview
The main focus of this lab is learning client-server socket programming and implementing a specific message protocol for clients and servers to talk to each other. You will implement two versions of a chat or talk client and server program to allow users on different machines to talk. You will be given a protocol for how clients and servers communicate, and your solution must implement the protocol such that your clients and servers can communicate with other group’s.
In this lab you will implement 2 different versions of the client and server.
-
The Part 1 version: allows for one-at-a-time chat session between a single client on one machine and a single server on another machine. When the current chat session ends, another client can connect to the server end and start a new chat session. The client and server alternate sending messages back and forth (in a question and answer type style).
The Part 1 solution is due in one week. I encourage you to finish this part early and get started in on Part2.
-
The Part 2 version: allows for a chat session between multiple clients. In this version the server does not participate in the chat itself, but fosters allowing multiple clients to connect to the chat session. Clients participating in the group chat send their message to the server and the server shares it with all of the other clients participating in the chat session. In this version, clients can "speak" at anytime; there is no specified ordering or taking turns writing messages to the group. Individual clients can also come and go in a chat session.
The Part 2 solution is due in two weeks.
This assignment is very loosely based on Unix talk, which was a very early Chat or IM system. (Note that you will NOT be implementing the Unix talk protocol in this lab; instead you will implement a different protocol that has some similar general functionality.)
Starting Point Code
-
Clone your Lab 2 repo from the CS87 git org:
git clone [your_Lab02_ssh_URL]
If all was successful, you should see the following files when you run ls:
ls README.adoc part1/ part2/
The Part 1 starting point code is in the
part1/
subdirectory:cd part1/ ls Makefile README.md client.c cs87talk.c cs87talk.h server.c
If this didn’t work, or for more detailed instructions on git see: Git Help Page
Starting Point Files
Part 1 starting point files are in the part1/
subdirectory. The
part2/
subdirectory is for your Part 2 solution (you will copy over
your Part 1 solution as the starting point to your Part 2 solution).
These are the starting point files for Part 1 (many already have some code written for you):
-
cs87talk.[c,h]
: definitions used by both the client and server, and common functions used by both the client and server side should go here. For example, both the client and the server send and receive messages on sockets. I recommend adding some generic message sending and receiving functions here, functions that will be used by both your client and server code. -
server.c
: the server side should be implemented here (there is a lot of starting point code for you in here) -
client.c
: the client side should be implemented here (there is a lot of starting point code for you in here) -
Makefile
: builds client and server executables (cs87_client, cs87_server). -
README.adoc
: some notes to you about code.
Running
To test your client and server, first start the server
(cs87_server
) on one machine, then start a client (cs87_client
)
on another machine, specifying the server’s IP address and the chat
session name as its command line arguments:
# on some machine start the server:
./cs87_server
# connect to the server on another machine
# list its IP address and your chat session name
# as command line options:
./cs87_client 130.58.68.67 tia
You can use the CS department machine specs page to find other machines to ssh into and run clients or servers.
See Tips and Resources for information on using nslookup
, dig
, and ifquery
to
find IP addresses.
Details: Message Protocol
A messaging protocol defines how two parties communicate for a particular service—the protocol defines both the set of application-level (higher level) message types, the send-recv exchange in handling each message type, and the expected format of messages.
For this lab you will implement the cs87talk protocol that has three different types of messages, corresponding to three different types of communication between cs87talk clients and cs87talk servers. The three are:
-
HELLO
: a client introduces itself to the server before talking. The server will tell the client if it can join the chat or not as part of this exchange. -
MSG
: an endpoint will send a message to another endpoint. A client can send aMSG
to a server and a server can send a MSG to the client. -
QUIT
: a client tells a server it is done and leaving the talk session.
Each part of the protocol begins with one party sending to the other party a 1 byte message tag. The tag value is used to indicate which of the three parts of the protocol the client and server should run (the expected send-recv sequence between the client and server for the interaction type is indicated by the tag).
Protocols Definitions
Each of the three parts of the cs87talk protocol are defined below. All message types begin by the sender sending a 1 byte message tag indicating the type of message.
As you implement these, make sure to look at the cs87talk.h
file for
type and constant definitions that you should use in your
implementation.
HELLO:
This message type is initiated by the client when it wants to start a talk session with a server (the client should initiate this immediately after the TCP connection has been established and before trying to send talk messages to the server). The send and receive behavior is defined by the following:
client server
====== ======
1. send HELLO_MSG tag -------------> recv 1 byte tag
2. send len -----------------------> recv 1 byte len
3. send name ----------------------> recv a len bytes name string
4. recv 1 byte tag <--------------- send 1 byte reply tag
(HELLO_OK or HELLO_ERROR)
The HELLO message starts with the client sending 3 values to the server, in this order:
-
the HELLO_MSG tag as a 1 byte value
-
a 1 byte len value, which is the number of bytes in the name string
-
a len bytes message containing the name string value (the string passed as a command line argument to the client program).
On receipt, the server decides whether or not to accept the client’s HELLO request (the decision algorithm is different for Part1 and Part2), and sends a response to the client indicating its decision.
The client then receives the 1 byte tag response from the server,
indicating if the server is letting it join the talk session
(HELLO_OK
:yes, HELLO_ERROR
:no).
If the server responds HELLO_ERROR
it should terminate its connection
with the client after it sends the HELLO_ERROR
message. If it
replies HELLO_OK
, it will update state about
this connection, and create internal state needed for communicating with
this client (allocate buffer space for messages, etc.). The server will
use the passed name string to prefix any string it creates from
subsequent messages received by this client with a "name: " to signify
that this message was "said" by client "name".
MSG:
This type of message is initiated either from a client to a server or from a server to a client, and is used to send talk messages between the two. Its defined by:
initiator other end
========= =========
1. send MSG_DATA ---------------> recv 1 byte tag value
2. send len ---------------------> recv 1 byte len value
3. send data --------------------> recv len byte message
If the server is the "other end" it should store the received message in a string with the a "name:" prefix (the name is obtained from the HELLO protocol) to identify the sender. For example, if the server is communicating with Freya, and Freya initiates a MSG_DATA message, sending "hello there" as the data part, the server will store the received message in a buffer (char array), ultimately creating the string: "Freya: hello there". Note that the resulting string printed out should not be longer than MSGMAX characters (and cannot be larger than 255 since this is the largest value that can be represented in 1 byte len). As a result, because the server adds a name prefix to the received message, the server may need to truncate the received message in its resulting string.
When a client receive a data message from the server, it just receives len bytes and creates a string from the len byte message (be sure to null terminate). the client side doesn’t add a prefix to strings created from received messages. For example if the server sends the client Freya a MSG_DATA message, with "hello there" as the data part, the client just stores the received message in the string "hello there", not in the string "Freya: hello there".
QUIT:
This message type is initiated by the client when it wants to drop out
of the chat session. The client sends the QUIT
message tag to the server
notifying it that it is leaving:
client server
====== ======
1. send QUIT ----------------------> recv 1 byte tag value
After sending QUIT
, the client should close its end of the socket and exit.
The server, on receiving a QUIT
message, cleans up any state associated
with this connection, including closing its end of the socket that was
dedicated to this communication.
After receiving a QUIT
, the server process should not exit.
-
For Part 1: it should be ready to accept a new client connection for a chat session.
-
For Part 2: this client has just left the group chat, which will continue with any other clients left in the chat session.
About the max data sizes and strings
-
Up to
NAMEMAX
bytes (chars) can be sent in the data part of aHELLO_MSG
message, and up toMSGMAX
bytes can be sent in aMSG_DATA
message. You can use these constant definitions to allocate message buffers to handle any size message one time in your program vs. on mallocing and freeing buffer space on every message send/received. (NOTE: your message buffer may be larger than these sizes in capacity…read on). -
The bytes sent in the
data
part ofMSG_DATA
andHELLO_MSG
should not include the final null terminating character (\0
) of a string. However, the null terminating character needs to added before printing out the message as a string. -
You can use the same char buffer to receive the message and to construct its resulting string. For example, in the server you can do this by receiving the
MSG_DATA
data bytes starting after the "name : " prefix at the start of the buffer--the message buffer can be larger thanMSGMAX
chars, and then adding the null terminating charater to string-ify the result. It is okay to use one buffer for receiving the message bytes and another to construct the string to print, but using just a single buffer for both is more efficient. In either case, you should not malloc and free message buffer and the resulting message string space on every message send and receive. -
The server may need to truncate a message it receives when it produces the resulting string to print out (Part 1), or to send to other clients (Part2) in order to ensure that the message’s total length is at most
MSGMAX
bytes. For example, if the client sends the server a message of lenMSGMAX
, the server should receive allMSGMAX
bytes of the message, but the server will then have to truncate off the last "name: " characters of the message in the resulting string that is printed out (Part 1) or sent to other clients (Part 2); it is important that the server (and client) can receive a full MSGMAX sized message, but the server may need to truncate off some characters at the end when creating the resulting string from the message bytes it receives.
Details: Part 1
Both Part 1 and Part 2 use the same messaging protocol, but the behavior of the
client and server is different in the two implementations. Your Part 1
solution should be implemented in the part1/
subdirectory of your Lab2 repo.
Your Part 2 solution should be implemented in the part2/
subdirectory of your
Lab2 repo.
Part 1 is an implementation where one client at a time connects to a server, and the server and client have a dedicated talk session.
In this version, a client connects to a server with the HELLO message, and then the client and server talk back and forth, each taking turns sending a message to the other. The server has a dedicated talk session with this client until the client terminates the connection (either it closes its end of the socket or it sends a QUIT message). The client process should exit after this, but the server should not exit. Instead, the server is now available to accept a connect from another client to have a talk session.
Specifically, the behavior should be:
-
client connects to a server using the TCP protocol. Run the client with the server’s IP and and the client’s chat name as command line args:
./cs87_client 130.58.68.67 tia
-
the client initiates the HELLO message part of the protocol with the server
-
if the server replies HELLO_ERROR, the client exits (no talking will take place)
-
if the server responds with HELLO_OK, then the client and server will start a talk session
-
-
the client and server have a talk session that consists of some number of alternating message exchanges. The first MSG should be sent from client to server, then next from server to client, the next from the client to server, and so on.
-
the talk session is ended by either the client sending a QUIT message or the server detecting that the client’s end of the socket has been closed.
An individual message in (3) should be a line of texted entered by the
user running the client (or server) side. I suggest using readline
to
read in a line of user input and return it as a string (and remember you
are responsible for freeing the return string when done with it…run valgrind).
If the user enters a string that is longer than the max message size, just
truncate the string sent to be just the first MAXMSG
len chars.
Sample Output
Here is sample output from two different Part 1 talk sessions with a talk server. Your client and server program can print out any prompts and messages to the user (they don’t have to be identical to mine), but they should have this same alternating behavior and the server should handle only a single client connection and talk session at a time. Also, each end should print out the message they received from the other (shown in bold in this example). Only after the current talk session ends, can the server accept the next talk session from another client (note where this happens is in the server output).
Details: Part 2
For Part2, you will start with your Part 1 solution, and change it
to implement a multi-client chat server. First, copy your part1
code into
your part2
subdirectory to use as the starting point for your Part2
solution:
cd part2 ls README.adoc cp ../part1/* . cp: overwrite './README.adoc'? n # n: don't copy over README.adoc ls Makefile README.adoc client.c cs87talk.c cs87talk.h server.c
The Part 2 talk server changes the role of the server, allows multiple clients to simultaneously connect to the same server to all participate in a group chat, and allows clients to send messages to others at any time (there is no predefined order or alternating behavior). In addition, new clients can enter an existing chat at any time, and current clients can leave at any time, and the chat session continues forever (until the server dies).
Part 2 Requirements
-
The server allows multiple clients to be connected at once.
-
The server is no longer a participant in the chat: it just facilitates some number of clients to have a group chat: the server doesn’t read in input from a user, nor does it echo any of the messages it receives from clients. Its only output should be a printing a message when a new client enters the chat and printing a message when a client leaves (or a connection ends).
-
The server must be multi-threaded: on each client connection (
accept
), the sever should create a new thread dedicated to that connection; when the client quits, the thread should clean-up any server state associated with its client, and then this server thread should exit. -
The main server thread, after an
accept
and creating a worker thread dedicated to the new connection, should go back to the main accept loop, ready to accept a connection from another client. -
The server should accept no more than 5 simultaneous client connections. If there are 5 clients connected and a new one tries to connect, the server thread should reply
HELLO_ERROR
to this client and terminate the connection with this client. When a connected client leaves, then the next connect request should succeed.#define
some constant for this max size (5), and use it in your code. -
When a server thread receives a message from one of its clients, it will send the message to all other clients.
-
The message sent will have the client’s name prefix.
-
The server should keep an array of structs of client info type you defined in Part1 so that the server thread can send a message on all client’s sockets. Think about if you need any synchronization.
-
-
The client should be multi-threaded with 2 threads:
-
one thread handles receiving
MSG
messages from the server. -
the other thread handles sending messages to the server from input that it reads in from the user.
sockets are bi-directional communication channels, which means that one client thread can send a message on the socket to the server, while another thread can receive a message from the server on the same socket.
-
-
Clients can connect and disconnect at any point
-
Clients can send messages whenever they read in a line entered by the user: there is no message ordering among the clients.
Sample Output
Here is sample output from a multi-client chat session. Note that the server is not a participant in the chatting, but instead forwards messages from one client to all others (and note how the the name prefix is included in the mesages from the server). Also note that clients enter and leave and the chat session continues.
A note about client-side I/O
Because you will have mulitple client threads sharing the same stdin and stdout, you will see messages from other clients being printed out in the same terminal window as the prompt and the input message the human user is typing in. This is fine.
Requirements
-
Use the TCP sockets client-server connection protocol. Here is a picture of the TCP client-server connection protocol:
-
Write your solution in C, using the starting point code.
-
Use the definitions in cs87_talk.h in your program. For example, use the type tsize_t rather than unsigned char, and use the #defines for message tags:
tsize_t tag; tag = HELLO_OK; ret = send(socket, &tag, sizeof(tsize_t), MSG_NOSIGNAL);
-
For Part 1, the server should never reply HELLO_ERROR (in Part 2 this is a valid option for the server to return).
-
Make sure to close sockets when done or when an error is detected.
-
Your solutions should be robust. Check return values from all system calls, and handle appropriately.
-
Use
send
andrecv
functions to send and receive messages on the TCP socket. For recv the flags field should be 0, for send pass MSG_NOSIGNAL to avoid getting sent a SIGPIPE signal when the other end of the connection closes its end of the socket. -
You should set the socket options SO_LINGER to off and SO_REUSEADDR on the server’s listen socket (see Tips and Resources).
-
Any message read in from the terminal entered by the user that is longer than MSGMAX should be truncated before being sent (make sure a sender never sends a message greater than MSGMAX bytes), and similarly for the NAMEMAX length limit.
MSGMAX is currently defined to be 255, and it is not possible to send a message size value that is larger than 255 (since len is passed as a single byte). However, your code should work regardless of what MSGMAX is defined, up to 255. For example, if it is redefined to be 32, your code for checking and only sending up to MSGMAX bytes should still work. -
You should define a struct on the server side for keeping track of a client’s information (this will be necessary for Part 2). The struct should minimally hold:
-
the client’s socket file descriptor (returned by call from
accept
). -
a message buffer (array of char) of some max size. You can statically or dynamically allocate this, but it should be allocated only once and reused over and over (think about what largest possible would be). I suggest copying the client’s "name:" to the front of this buffer so it will be a prefix of all messages from this client.
Remember, however, that every message sent can be no more than MSGMAX bytes. This means that if a client’s message length plus the name prefix length is longer than MSGMAX, the server needs to truncate the client message before printing out (Part 1) or sending to another client (Part 1). For example, if the prefix is "Tia:" then there are 4 fewer characters to store a message. If the prefix is "Kevin:" then there are 6 fewer. You should also have space in the buffer for the null terminating character of the string '\0' ('\0' shouldn’t be passed with the other bytes of a message). As a result, for Part 2, a client can send a long message that the server may have to truncate to send to other clients because the server adds the sending client’s name as prefix to each message it sends.
-
where the start of buffer is for message data (i.e. the first index into the message buffer for the start of the mesage after the name prefix).
-
-
Because you are all implementing the same protocol, your client should be able to connect to another group’s server and have a chat. Your server should be able to accept a connection from another group’s client and have a chat-- all Part1 solutions should be compatable, and all Part2 solutions should be compatable. We will test this out in lab next week, but try it out with other groups in the lab as you work on this too.
-
Check return values from all system and function calls and handle errors. For system calls (like send, recieve, socket, etc.), you should call
perror
to print out an error specific message. Here is one example of how I might call perror and exit if a call tosetsockopt
fails:int reuse_true = 1; retval = setsockopt( listenfd, SOL_SOCKET, SO_REUSEADDR, &reuse_true, sizeof(reuse_true) ); if (retval < 0) { perror("Setting socket option SO_REUSEADDR failed"); exit(1); }
Extra Features
Extra Features are NOT required, but are some ideas for extra things to add
to your Part2 solution to make your chat program a little more
user friendly. If you try any of these, please first create a new subdirectory
extrafeatures/
and copy your part2/
solution into it and modify this
version and not your part2
:
cd ~/cs87/Labs/Lab02-you-partner/ mkdir extrafeatures cd extrafeatures pwd cp ../part2/* . ls
Here are some suggestions for extra features to add:
-
When a new client connects to a group chat session, the server send all other clients a message saying that they have joined (like "Freya has joined the chat").
-
When a client leaves the group chat session, the server sends all other clients a message saying that the user has left the chat session (like "Freya has left the chat")
-
When a client sends a chat message to the server, the server sends the message to every other client except for the client whose message it is. For example, if Jo, Flo, and Bo are in a chat and Bo types in a message, the server sends Bo’s message just to Jo and Flo.
-
Add to the protocol a new message type for a client to ask the server for all the names of people in the chat session. The user on the client side would have to type in a specific string to trigger this protocol (like typing in "goodbye" triggers the QUIT protocol), this would send a specific message tag to the server that would have it send back to the client the names of all other chat members.
-
Add to the protocol a new message type for a client to ask the server to send it all of the last messages from each chat member. Again, the user on the client side would trigger this by entering some unique string to distinguish this request from a message to share with other clients in the chat session.
-
If you want to try to implement a solution with nicer looking output, you can try using the ncurses library to split the terminal window and direct stdin to one half and stdout and stderror to the other half (this is not trivial to do). You could also try something easier and print out stdout output in a different color if it is coming from the prompt printing thread vs. from the thread that receives and prints out messages from other clients forwarded from the server. Here is some information about ncurses: Cool C libraries
Submitting
Before the Due Date, one of you or your partner should push
your solution to github from one of your local repos to the GitHub
remote repo. Be sure to do a make clean
before you git add
anything.
For example, to submit Part 1, do:
cd ~/cs87/Labs/Lab02-me-you/part1
make clean
git add *.c *.h
git commit
git push
git status
Make sure your Part 1 solution in in the part1
subdirectory, and your
Part 2 solution in in your part2
subdirectory (this is where we will
look to grade your solutions).
If your files are not in the right place, use the mv
command to move files
from one location to another. For example, if you accidentally put your Part 2
solution in the top-level repo directory, you can move it into part2/
like
this:
pwd /home/me/cs87/Labs/Lab02-me-you make clean mv Makefile part2/. mv *.[c,h] part2/. cd part2/ # test it out to be sure all the files are in part2 subdirectory make ...
See the git help page "Troubleshooting" section for git help.
Tips and Resources
socket programming
-
I have some Socket Programming Links.
Beej’s Guide is a good staring point and has code examples (sections 5 and 6 are particularly useful).
-
The Steven’s Unix Network Programming (2nd Edition Vol. 1). Chapters 3 and 4 are very helpful. We have a couple copies of this in one or more CS Lab spaces. Please don’t remove these from the CS labs.
-
You should use
send
andrecv
to send and receive messages on sockets.send
andrecv
should be called in a loop until all the bytes of the message have been received. If a call to send or recv returns with fewer bytes than were send/recived, you need to call again to send/recvie the remaining bytes of the message, starting from within the buffer you are sending/receiving. For example, you can specify the next point to receive in a buffer by passingrecv
the address of the bucket:ret = recv(sock, &(buf[i]), amt_left, 0);
-
set sockopts on the listen socket: SO_REUSEADDR and SO_LINGER. If you set the sockoption SO_LINGER to off, then the socket will close immediately upon a process exit (This is done for you already in the starting point code, just understand what it is doing and don’t remove it):
struct linger linger_val; linger_val.l_onoff = 0; linger_val.l_linger = 0; setsockopt(sock_fd, SOL_SOCKET, SO_LINGER, (void *)(&linger_val), (socklen_t) (sizeof(struct linger));
-
Read the man pages for system calls like socket, connect, accept, listen, send, recv, etc.
man 2 socket
Note RETURN values and be sure to check for error returns and handle appropriately in your code. You should use
perror
to print error return messages from system calls like these, andprintf
for non-system call functions that may return error values (like error return values from function you write). -
use perror to print out error messages from failed system calls:
int *sock_fd; sock_fd = socket(AF_INET, SOCK_STREAM, 0); if(sock_fd == -1) { perror("socket create failed. Exiting\n"); exit(1); }
Look at the RETURNS part of the man pages of system calls to see what they return on success or error.
-
run
/sbin/ifquery eth0
to get your own IP address.$ /sbin/ifquery eth0 # look at the address entry for the IP address
-
nslookup or dig to get another machines IP address
$ nslookup lime Address: 130.58.68.165 $ dig +short lime.cs.swarthmore.edu 130.58.68.165
C, strings, readline
-
Chapter 2 of Dive into Systems covers C programming, structs, arrays and strings, pointers, pass-by-pointer parameters, switch statment, dynamic memory allocation, and more. And here are some other C programming references.
-
In particular, take a look at C strings and the C string library functions parts. This program involves C string manipulation for reading in a message from the user, constructing a message to pass from one party to another, and for printing out messages recieved. The
strncpy
function may be useful. You also should not be sending the terminating\0
in messages. As a result, you will need to add it to the end of any string you want to print.
-
-
readline library. You can use readline to read in messages entered by the user in the client. It is possible that the user types in a message that is too long to send in a MSG message. In this case, truncate the entered message to the max size that will fit.
-
fflush
: to force printf to output printf output is buffered and may not show up to the terminal until the program has executed many instructions past the printf stmt. To force printf output to stdout, you can call fflush:printf("hello there"); fflush(stdout); // force all buffered printf output to stdout
-
valgrind and gdb guides. Also see Chapter 3 of Dive into Systems on debugging C programs.
pthreads
-
Chapter 14 of Dive into Systems covers pthread programming.
-
Links to other pthreads documentations and tutorials off my help pages.
-
Some tips for debugging pthread programs, or debugging pthread programs from Dive into Systems chapter 3, and more details here: guide for using gdb to debug pthread programs.
misc help pages
-
my help pages includes other links to Unix, C, pthreads, networking resources
-
Dive into systems(C programming, and Appendix 2 on Using Unix)
-
man pages for C library, pthreads library, and TCP/IP socket functions.
-
Tools for examining system and runtime state top and htop to see what is executing on client and server side
-
some useful tools vim, ssh, scp, tmux, finding idle machines, latex, git, …