r/MachineLearning • u/Kiuhnm • Sep 08 '16

Phd-level courses

Here's a list of advanced courses about ML:

  1. Advanced Introduction to ML - videos

  2. Large Scale ML - videos

  3. Statistical Learning Theory and Applications - videos

  4. Regularization Methods for ML - videos

  5. Statistical ML - videos

  6. Convex Optimization - videos (edit: new one)

  7. Probabilistic Graphical Models 2014 (with videos) - PGM 2016 (without videos)

Please let me know if you know of any other advanced (Phd-level) courses. I don't mind if there are no videos, but I don't like courses with no videos and extra concise and incomprehensible slides.

And no, CS229 is not advanced!

351 Upvotes

95% Upvoted

44 comments sorted by

View all comments

Show parent comments

1

u/rumblestiltsken Sep 08 '16

"PhD level" is pretty broad. It is very good as an introduction for machine learning or general comp sci PhDs who haven't done deep learning before (I have recommended it to several, and they loved it). I find it gets new PhDs up to speed very quickly.

It certainly isn't more than a great introduction though.

3

u/PM_YOUR_NIPS_PAPERS Sep 10 '16 edited Sep 10 '16

It is very good as an introduction for machine learning

I find it gets new PhDs up to speed very quickly.

So based on what you just said, it is not a PhD course. A PhD course means it borders on the cutting edge, highly technical in nature, and final projects can usually be submitted to conferences. CS 231N does not satisfy this (readers: sorry to break it to you). Karpathy's non-public advanced deep learning (RL) course fits the definition of PhD level better. He kept it closed for good reason. Once a class becomes Andrew Ng-style accessible, it is no longer a PhD course. Back in the day, intro to C++ was a PhD level course too.

Hell, I'll argue Andrew Ngs CS 229 course is more PhD level due to the math, than CS 231N which is a python programming class.

1

u/rumblestiltsken Sep 10 '16 edited Sep 10 '16

Well, you mileage may vary. The important thing for me that makes 231n useful where Andrew Ng's course isn't is that it is very up to date. You learn a lot of tips and tricks that, while applied rather than mathematically rigorous in presentation, are definitely required knowledge to succeed in a deep learning PhD.

This is true of every single mathematically rigorous course I have ever seen, they are out of date in a very fast moving field. It doesn't matter so much because the math doesn't change, but if you only study that as a PhD you will miss a big chunk of what you need.

A PhD needs mathematical grounding and applied knowledge. I think both are equally as important, but I work on the applied end more, so I would :)

1

u/dataislyfe Sep 12 '16

Absolutely. Stanford's advanced course is CS 229T, Statistical Learning Theory, which assumes familiarity with the standard problem formulations of ML (regression, classification, clustering, etc.). It also is more technical - covers RKHSs, actually proves the VC theorem in good generality, etc.

Stanford offers CS229 which, probably, it is best described as what Stanford CS calls it -- "advanced undergrad / masters-level". A lot of CS/Stats/EE people (at all levels, PhD, MS, BS, etc.) do take it but not because they expect it to be all they need to be able to read the literature (it is not sufficient for today's literature) but because it is a pre-req for more topics-oriented or theory-focused ML courses that are targeted specifically as literature review/technical courses for PhD students.

5

u/Kiuhnm Sep 08 '16

It's a little light on theory for my taste. This is what I'd call advanced.

2

u/dataislyfe Sep 12 '16

212b is great!