Week 11: RNN

Announcements

  • Lab 10 is due 04/05 by 23:59 EST.

  • Final Paper is due 05/10 by 23:59 EST.

  • Lab 11 will be available Wednesday, 04/06. It is a team assignment.

  • Lectures and labs this week are In Person, office hours are via ZOOM.

  • Lectures video recordings are in our shared Google folder.

  • Class participation, EdSTEM, Figma, and the Group 2 of The six groups for lecture notes.

  • 2_wk_2_and_11, aburges1, amounts1, xwang5, vzakhar1, bxiang1

  • Austin, Amelia, Xuduo, Vika, Brian

  • (Back to Course Index Page)

Week 11 Topics

  • RNN

  • Related Paper

  • Related Course

  • Related Math

  • Related Code

Monday

Guest Speaker

  • Professor Ameet Soni

  • Past Courses

  • CPSC 66 - Machine Learning (Spring 2021,Fall 2017)

  • Research Interests: Machine Learning and Biomedical Applications

  • 2017 Faculty Lecture for an overview of his research.

  • EAAI Panel: Teaching AI Ethics

  • Sheila McIlraith, Emanuelle Burton, Ameet Soni, Trystan Goetze

Recurrent Neural Networks (RNNs)

  • What’s your understandings of RNNs?

  • Neurons with Recurrence

  • RNN Intuition

  • Networks with loops

    1. Apply a recurrence relation at each step

    2. input vector

    3. output vector

    4. RNN between input and output

  • GRU (Gated Recurrent Units)

  • LSTM (Long Short-Term Memory)

  • RNN Tricks

    1. Clipping gradients

    2. Initialization

    3. LSTM self-loops

Wednesday

Students Presentations

  • What’s your understandings of RNNs?

RNN and Time

  • Long term, short term

  • Forget

  • Time-series

  • Sequence Modeling

RNN Implementation

  • Paper

  • Toolset

Friday

Students Presentations

  • What’s your understandings of RNNs?

Attention

Transformer

  • Paper

  • Toolset