CS 66 Lab 6

Due Monday, 10/24/2022, by midnight (23:59, EST)

Announcements

Goals

The goals for this lab assignment are:

  • Learn how to conduct Literature Review

  • Practice how to review research papers

  • Get comfortable with using reviewer guide from CS conferences

  • Get comfortable with ten-page research papers

1. Reviewer Guide

For this week, please implement the below two sections of the Reviewer Guide:

  • Review content

  • Examples of Review Content

2. Review Examples

Please read the two paper review examples below:

8-8-10, Best paper award Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research By Bernard Koch, Emily Denton, Alex Hanna, and Jacob Gates Foster.

7-7-10, ML and BCI EEGEyeNet: a Simultaneous Electroencephalography and Eye-tracking Dataset and Benchmark for Eye Movement Prediction By Ard Kastrati and Martyna Beata Plomecka and Damian Pascual and Lukas Wolf and Victor Gillioz and Roger Wattenhofer and Nicolas Langer.

3. Write Your Own Review

  • Review the FIVE midterm papers you are assigned in midterm folder

  • You will get a reply email to your midterm submission about which five posters you are required to review.

  • Follow the instructions about Double-blind reviewing on the Reviewer Guidelines page.

  • Write your review based on the Reviewer Guide and Review Examples above. You should include:

    1. Summary and contributions: # At least five sentences, including its innovation, connection, and impact.

    2. Strengths: # At least five sentences, with details from the poster to support your opinion.

    3. Weaknesses: # At least eight sentences, with details from the poster to support your opinion. 'Your comments should be detailed, specific, and polite. Please avoid vague, subjective complaints. ' 'Always be constructive and help the authors understand your viewpoint, without being dismissive or using inappropriate language. '

    4. Correctness: # At least one sentence.

    5. Clarity: # At least three sentences, examples are required 'Give examples of what parts of the paper need revision to improve clarity.'

    6. Relation to prior work: # At least three sentences, 'The related work section should not just list prior work, but explain how the proposed work differs from prior work appeared in the literature.'

    7. Reproducibility: # At least five sentences, example required 'Mark whether the work is reasonably reproducible.'

    8. Additional feedback: # Optional.

    9. Overall score: # At least one sentence, I expect 6 out of 10 or lower, compared to the EEGEyeNet (7-7-10). '6: Marginally above the acceptance threshold.' The rubric focuses on how you support your peers to improve their papers. This overall score is purely for improving your peers' posters, NOT for the midterm scores.

    10. Confidence score: # At least one sentence.

    11. Broader impact: # At least two sentences.

    12. Ethical concerns: # At least one sentence.

  • The '#' parts above are the comments for each section, similar to comments in Python.

4. Submission Guide

  • Each student team only submits one file, lab_6_lastname1_lastname2.zip, including

    1. Five review PDF files in AAAI format, such as paper_1_reviewer_A7D5.PDF

    2. Keep the review process double-blind, your team will be assigned a team number and a reviewer code.

5. Notes

  • Email 'xqu1@swarthmore.edu' your zip file for lab 6, and cc your teammate.

  • Lab assignments will typically be released on Tuesday and will be due by midnight on the following Monday.