Name | Mike Izbicki (call me Mike) |
mizbicki@cmc.edu | |
Office | Adams 216 |
Office Hours | Monday 9:00-10:00AM, Tuesday/Thursday 2:30-3:30PM, or by appointment (see my schedule); if my door is open, feel free to come in |
Webpage | izbicki.me |
Research | Machine Learning (see izbicki.me/research.html for some past projects) |
Fun Facts | grew up in San Clemente, 7 years in the navy, phd/postdoc at UC Riverside, taught in DPRK |
This is a course on deep learning (not big data).
Course Objectives:
Learning objectives:
- Write basic PyTorch applications
- Understand the "classic" deep network architectures
- Use existing models in a "reasonable" way
- Understand the limitations of deep learning
- Read research papers published in deep learning
- Understand what graduate school in machine learning is like
- (Joke) Understand that Schmidhuber invented machine learning
My personal goal:
- Find students to conduct research with me
Expected Background:
Necessary:
- Basic Python programming
- Linear algebra
- Calc III
- Statistics
Good to have:
- Machine learning / data mining
- Lots of math
- Familiarity with Unix and github
Resources:
Textbook:
- The Deep Learning Book, by Ian Goodfellow and Yoshua Bengio and Aaron Courville; I will assume that you already know all of Part I of this book (basically the equivalent of a data mining/machine learning course)
- Various papers/webpages as listed below
Deep learning examples:
-
Images / Video
-
Text
-
Games
-
Other
The good:
- most influential research in 2019 is deep learning papers
- /r/machinelearning
- recent open source AI programs
- The state of jobs in deep learning
- The decade in review
The bad:
- Machine learning reproducibility crisis
- Logistic regression vs deep learning in aftershock prediction
- Pictures of black people
- NLP Clever Hans BERT
- Ex-Baidu researcher denies cheating at machine learning competition
Computing resources:
- Google Colab provides 12 hours of free GPUs in a Jupyter notebook
- Kaggle provides 30 hours of free GPU
- I have a 40CPU/8GPU machine that you can access for the course
- I have another 4CPU/1GPU machine that needs someone to set it up
Videos:
Week | Date | Topic |
---|---|---|
1 | Tues, 21 Jan | Intro: Examples of Deep Learning |
1 | Thur, 23 Jan | Automatic differentiation |
2 | Tues, 28 Jan | Machine Learning Basics (Deep Learning Book Part 1, especially chapters 5.2-5.4) |
2 | Thur, 30 Jan | Optimization |
3 | Tues, 04 Feb | Image: CNNs |
3 | Thur, 06 Feb | Image: CNNs II Summer Research |
4 | Tues, 11 Feb | Regularization |
4 | Thur, 13 Feb | Image: ResNet More links: |
5 | Tues, 18 Feb | ResNet continued |
5 | Thur, 20 Feb | ResNet continued
|
6 | Tues, 25 Feb | YOLO The MvMF loss for geolocation: |
6 | Thur, 27 Feb | Text: Basic text modelsPython text processing libraries: |
7 | Tues, 03 Mar | Text: CNNsText: RNNs |
7 | Thur, 05 Mar | Text: Lab exercise |
8 | Tues, 10 Mar | Text: Seq2seq |
8 | Thur, 12 Mar | Text: Attention |
9 | Tues, 17 Mar | NO CLASS: Spring Break |
9 | Thur, 19 Mar | NO CLASS: Spring Break |
10 | Tues, 24 Mar | Text: Transformers (paper, blog post) |
10 | Thur, 26 Mar | TBD |
11 | Tues, 31 Mar | TBD |
11 | Thur, 02 Apr | TBD |
12 | Tues, 07 Apr | TBD |
12 | Thur, 09 Apr | TBD |
13 | Tues, 14 Apr | TBD |
13 | Thur, 16 Apr | TBD |
14 | Tues, 21 Apr | TBD |
14 | Thur, 23 Apr | TBD |
15 | Tues, 28 Apr | TBD |
15 | Thur, 30 Apr | Project Presentations |
16 | Thur, 05 May | Project Presentations |
16 | Thur, 07 May | NO CLASS: Reading Day |
Week | Weight | Topic |
---|---|---|
2 | 10 | Rosenbrock Function |
3 | 10 | Crossentropy Loss |
4 | 10 | CNN |
6 | 10 | Image Transfer Learning |
7 | 10 | RNN |
10 | 10 | Text Transfer Learning |
-- | 10 | Reading |
15 | 30 | Project |
There are no exams in this course.
Late Work Policy:
You lose 10% on the assignment for each day late. If you have extenuating circumstances, contact me in advance of the due date and I may extend the due date for you.
Collaboration Policy:
You are encouraged to work together with other students on all assignments and use any online resources. Learning the course material is your responsibility, and so do whatever collaboration will help you learn the material.
I want you to succeed and I'll make every effort to ensure that you can. If you need any accommodations, please ask.
If you have already established accommodations with Disability Services at CMC, please communicate your approved accommodations to me at your earliest convenience so we can discuss your needs in this course. You can start this conversation by forwarding me your accommodation letter. If you have not yet established accommodations through Disability Services, but have a temporary health condition or permanent disability (conditions include but are not limited to: mental health, attention-related, learning, vision, hearing, physical or health), you are encouraged to contact Assistant Dean for Disability Services & Academic Success, Kari Rood, at disabilityservices@cmc.edu to ask questions and/or begin the process. General information and the Request for Accommodations form can be found at the CMC DOS Disability Service’s website. Please note that arrangements must be made with advance notice in order to access the reasonable accommodations. You are able to request accommodations from CMC Disability Services at any point in the semester. Be mindful that this process may take some time to complete and accommodations are not retroactive. It is important to Claremont McKenna College to create inclusive and accessible learning environments consistent with federal and state law. If you are not a CMC student, please connect with the Disability Services Coordinator on your campus regarding a similar process.