How to submit

In the ProjectProposal part of your Project repo, use the .tex and .bib files to write your proposal. (As soon as you have your project group, email me your project group members' names, and I will created a repo for you with the latex starting point files for your proposal and annotated bibilography). Run make in your ProjectProposal on a CS lab machine to build the .pdf of your proposal with annotated bibiligraphy document (it needs to compile with latex installed on our system as this is how I will build it). Look over the .pdf to make sure it is correct (you can use scp to copy it from the CS lab machines if you are building it there):

# from your home machine, in a shell:
scp you@cs.swarthmore.edu:./cs87/Project/ProjectProposal/proposal.pdf .

Once the .pdf looks fine, then run make clean to clean up compiled files, and do a git add of proposal.tex and proposal.bib and commit and push to your repo.

I will not accept any proposals late; a late submission will result in a zero for this part of the project.

These parts count towards 5% of your final grade

If you have not already done so, first read the page about the course project that gives a high-level overview of the project and lists the specific parts with tentative due dates for later parts: CS87 course project

Contents

• Annotated Bibliography
• Written Proposal updated with changes for 2020
• Requirements regarding Platforms and Languages
• Getting Started and Some Project Ideas

If you are doing a original style project, then you should create an annotated bibliography of at least 3 papers closely related to your project (more is fine).

If you are doing the second style of project, in-depth report, your annotated bibliography should have at least 4 papers closely related to your project (and I encourage a couple more).

As you read each paper, add an entry to your annotated bibliography. An annotated bibliography will list each paper as it would appear in a references section of a research paper. In addition, each listing is annotated with a couple paragraphs that describes the work. Use the course reaction notes as a guide for writing the annotation part: each paper should have 1 paragraph that summarizes the work, lists strengths and weaknesses, and discusses the main contribution of the work, and it should have 1 paragraph that analyzes the work in the context of how it is related to your project (think about asking and answering some questions about this work related to the project you are addressing).

For your own purposes, I encourage you to add a second part that lists additional related papers/work in your bibliography and to include a very brief summary of them (you do not need to include the full required annotation of the 3 most related). I will not grade these, but doing so may be helpful as you refine your project over the course of the semster, and it may help me point out some other related work to your project.

Look at the annotated bibliography section of my CS Research and Writing Guide the "Getting Started" section below for links to places to look for related work papers.

Your project proposal should be 4-5 pages long (in addition to your annotated bibiliography). It should clearly state the problem you are solving, how you are solving it, and explicitly list your plan for completing your proposed project.

You must use my latex template for your written report, it is in the ProjectProposal subdirectory of your Project repo.

Details about changes to the project types are here: changes for 2020

Original Type of Project Proposal (Type 1)

Specifically, for an orginal type of project your proposal should contain the following:

1. Title and Authors (include "Project Type 1" in your title)
2. An Introduction: 1-2 paragraph summary of the problem you are solving, why it is interesting, how you are solving it, and what conclusions you expect to draw from your work.
3. Related Work: 1-2 paragraphs describing similar approaches to the one you propose. This need not be an exhaustive summary of related literature, but should be used to put your solution in context and/or to support your solution. This is also a good way to motivate your work. This can be a summary taken from your longer annotated bibliography.
4. Your Solution: 3-4 paragraphs describing what you plan to do, how you plan to do it, how it solves the problem, and what types of conclusions you expect to draw from your work.
5. Experiments: 1-3 paragraphs describing how you plan to evaluate your work. List the experiments you will perform. For each experiment, explain how you will perform it and what the results will show (explain why you are performing a particular test).
6. Equipment Needed: 1 paragraph listing any software tools that you will need to implement and/or test your work. If you need to have software installed to implement your project, you should check with Jeff Knerr to see if it is something that can be installed on the CS lab machines.
7. Schedule: list the specific steps that you will take to complete your project, include dates and milestones. This is particularly important to help keep you on track, and to ensure that if you run into difficulties completing your entire project, you have at least implemented steps along the way. Also, this is a great way to get specific feedback from me about what you plan to do and how you plan to do it.
8. Conclusions: 1 paragraph summary of what you are doing, why, how, and what you hope to demonstrate through your work.

In Depth Report Project Proposal (Types 2 and 2b)

Specifically, for an in-depth report type of project your proposal should contain the following:

1. Title and Authors (include "Project Type 2 or 2b" in your title)
2. An Introduction: 1-2 paragraph summary of the problem you are investigating, why it is interesting, what the main foci of the problem you are investigating, and what conclusions you expect to draw from your investigation.
3. Related Work: 1-2 paragraphs describing main related work to your project, to both the breadth and particular depth parts you plan to investigate. This need not be an exhaustive summary of related literature, but should be used to put your investigation in context and/or to support your plan. This can be a summary taken from your longer annotated bibliography. This section will be a much larger part of your final report than the Type 1 project may have for their related work.
4. Breadth: 2-3 paragraphs describing your plan for the breadth of coverage of your report. What are the big topics and big themes you plan to cover? How broad is the scope of your report (e.g. are you focusing on a specific algorithm, a family of algorithims, specific types of parallelization, ...)? What and why are contraints on its breadth? What are some high-level details of each of these big topics you plan to investigate (think about some refinement of a outline of this section with some information about the different parts of the breadth of your report)?
5. Depth: 2-3 paragraphs describing specific parts that you plan to investigate in depth, and your plan for what about these specific parts you will investigate. Unless your project does not fit this (and tell me why), I suggest that you have a one main depth part to your report for each member of your group. This way, each of you is primarily focusing on one of the in-depth parts of your project. You can additionally add others, and you can all help with each one, but this is a good way to plan how to distribute your in-depth effort among group members.
6. If you are doing project type 2b: 1-2 paragraphs about the experiements or testing you plan to do.
7. Schedule: list the specific steps that you will take to complete your project, include dates and milestones. If you are dividing up parts, specify who is doing what. This is particularly important to help keep you on track, and to ensure that if you run into difficulties completing your entire project, you have at least completed steps along the way. Also, this is a great way to get specific feedback from me about what you plan to do and how you plan to do it.
8. Conclusions: 1 paragraph summary of what you are doing, why, how, and what you hope to learn through your work.

You must use my latex template for your written report, it is in the ProjectProposal subdirectory of your Project repo. Once you notify me of your project group members, I'll create a git repo for your project that will include the latex template for your report. It will be based on one like this example: /home/newhall/public/latex_examples/paper
Your Project repo contains:

FinalReport/  MidwayReport/  ProjectProposal/  README.md  source/

You should write your proposal and your annotated bibilography in the ProjectProposal subdirectory in the proposal.tex file. Look at the README.md files for more information about the repo contents.

You are strongly encouraged to come up with your own idea for a course project. However, I have suggestions for some, which are available here (and please, these are not for public distribution):

evince ~newhall/public/cs87/project_ideas_s20.pdf

You are welcome to use one of these as a starting point for developing a project, or to come up with your own idea.

Once you have one or two general ideas for projects, you will want to more completely define exactly what you plan to do and how you plan to do it. A good way to start with this step is to take a look at related work; you want to have an understanding of what has been done, and how what you want to do fits into the field. It will help you modify and more completely define your project, and get some ideas of how to implement and test your project.

Take a look at the "Getting Started" section of my CS Research and Writing Guide for links to places to look for related work papers. There are links to USENIX conference proceedings and to ACM and IEEE digital libraries. I have links to some other (older) papers here: Additional Cluster and Distributed Computing Papers.

Another place to start getting ideas is to look at recent conference proceedings from SC, IPDPS, and other parallel and distributed computing conferences:

• SC conferences (look for Technical Session to find papers):
• IPDPS conferences
• Other ACM, IEEE, and USENIX sponsered conferences. Here are a few suggestions: PODC (Principles of Distributed Computing) www.podc.org, HPDC www.hpdc.org, ICDCS, PPoPP, IEEE Cluster, ... USENIX conference proceedings USENIX has a lot of top systems conferences. Some that may be more applicable to course projects include: OSDI, IPTPS, NSDI, HotPar, USENIX ATC, HotCloud, HotStorage, FAST
• other ACM sponsered conferences can be found of their main web page www.acm.org. Looking as specfic SIG pages may be useful, for example SIGOPS sponsored conferences: sigops confs
• all IEEE Conferences: www.computer.org and search for parallel and distributed conferences under the "Conferences" menu
• and USENIX conferences at www.usenix.org
  usenix proceedings links to all USENEX conferences and papers available.

Your project should compile and run on our system. You may use other systems too, like XSEDE, but it also should work on our system. If you use Amazon AWS, then Spark code parts of your project may not run on our system. Other than this, the expectation is that you can compile, run and demo your project to me on the CS lab machines.

If you need software installed on our system, contact Jeff Knerr.

You are welcome to use any language you'd like to implement your project. However, I strongly recommend that you use C or C++ to implement the bulk of it. Python may seem like an atractive language option, and it is great for writing scripts, however C and C++ are the languages for parallel and distributed computing (think MPI, CUDA, OpenMP, pthreads, ...). Using them will make things easier for you in the long run. If you don't beleive me, here is a quote about this from a student who took CS87 in 2018:

"We used python in our CS87 project, and we definently should have used C or C++. Python often has libraries that wrap around the parallelization methods you have been using, but they are usually not super well maintained. For instance, mpi4py works great until you try to spawn some relatively low number of processes (I think around 1000 or so), which would be allowed in native MPI. We couldn't figure out why, and the library isn't actively maintained, so we spent a lot of time working around this issue that could have been used elsewhere. Also, we got to a point where we wanted to combine MPI and multithreading for shared memory on node machines. Python doesn't support this feature in an efficient way like C or C++ would, and this limited the scope of our ideas. I highly recommend doing your final project in C or C++."

Machines and Software Available for your use:

• openMPI: runs on all lab machines
• openMP: on all lab machines
• CUDA: runs all lab machines
• all CS lab machines
• Subset of CS lab machines (ssh access only) available only to CS87 students:

  8 core: cardamom crabby cucumber elderberry hyssop mint parsley stew
  zucchini (12 core)
  chervil (16 core, 32 hyperthreaded cores)
  

• XSEDE: Use this only for large experiments after testing on other systems. Comet has both gpu and compute nodes. All machines support MPI, and many also support CUDA and other parallel languages. There are a lot of libraries installed to use too. The Comet User Guide has more information. Contact Andrew Reuther in ITS to get access to other XSEDE machines, or if you need more XSEDE allocation hours.
• The Swarthore Stelka Compute Cluster.
• Through ITS Swarthmore students get $100 credit to use on Amazon AWS. AWS runs Spark (Amazon's MapReduce) plus other cloud service software.

If there is other software you need that is not available on our stysem, we may be able to install it or install it on our system or on some of our machines. You should talk with me and Jeff about this.

Some more links: my list of parallel and distrubuted computing links