Resources I found useful as 1st year PhD student
Published:
Resources that I found useful in my first year of PhD.
During my EECS studies, I didn’t get an introduction to many theoretical topics that are both interesting and extremely useful for my research. In this post I will provide some links to some of the most useful resources. Some of the courses I did not finish, or I did not need everything that was presented, but I found them very didactic.
To understand Differential Privacy and Machine Learning, the foundational understanding of probability must be pretty sound. I only had a measure-theoretic introduction to probability in the first year of PhD, but I recommend getting one as soon as possible. Afterwards, everything makes so much more sense. I have mixed multiple resources when learning measure-theoretic probability, but the most fundamental ones were:
- Stanford CS109 - A good refresher on probability
- The Bright Side of Mathematics Measure Theory course: A nice introduction to measure theory and why to study it. My way of learning is to understand first why we must study something. This course covers all the whys well.
- Advanced probability course by Professor Lanchier: An introduction to measure-theoretic probability with some simple examples and some of the essential proofs. Easy to follow and did not scare an engineer.
- Measure Theory by Claudio Landim: A more detailed introduction to measure theory.
The basic and the more advanced things I was studying would sometimes reference ideas from functional analysis, which I never had a course on. Looking back, I would first take a course in functional analysis and then study measure theory. While the notations/way of doing this is different, everything made sense in measure theory after studying functional analysis as well.
- The Bright Side of Mathematics Functional Analysis couse
- 18.102 MIT - Still working on it, a fantastic course.
Now, we can wander into ways of applying these new tools. Some courses I have watched and enjoyed:
- Markov Processes by J.N Corcoran: Easy to follow, introducing a lot of the fundamental tools in applied probability. Professor Corcoran is a great teacher!
- Mathematical Statistics by J.N. Corcoran: Easy to follow, introduces estimators, confidence intervals, binary hypothesis, etc. The major downside of this course is that some courses need to be added.
- High dimensional probability by R. Vershynin: Great course to understand plenty of tools in high dimensional probability. Maybe too much was covered, but it was fascinating, nonetheless.
- Learning Theory by S.B. David: For learning theory people this course needs no introduction. Great introduction to theoretical machine learning.
- Large Scale Machine Learning by R. Salakhutdinov: Another great course in theoretical machine learning.
I never had an introduction to information theory in my undergrad either, yet some notions from IT emerged in machine learning and differential privacy. I followed this course and enjoyed it:
In the end, I will add the courses that study some of the things I am interested in:
- Differential Privacy by G. Kamath: One of a kind introduction in differential privacy!
- Sketching Algorithms by J. Nelson: Highly didactic introduction in sketching techniques.
I found out after starting my PhD that a significant component of the PhD life is writing. Here are some resources that I found helpful in writing:
I am a big fan of Oded Goldreich. His books and blog posts are a mine of gold, yet I will only highlight two of them