Resources I found useful as 1st year PhD student

Resources that I found useful in my first year of PhD.

During my EECS studies, I didn’t get an introduction to many theoretical topics that are both interesting and extremely useful for my research. In this post I will provide some links to some of the most useful resources. Some of the courses I did not finish, or I did not need everything that was presented, but I found them very didactic.

To understand Differential Privacy and Machine Learning, the foundational understanding of probability must be pretty sound. I only had a measure-theoretic introduction to probability in the first year of PhD, but I recommend getting one as soon as possible. Afterwards, everything makes so much more sense. I have mixed multiple resources when learning measure-theoretic probability, but the most fundamental ones were:

• Stanford CS109 - A good refresher on probability
• The Bright Side of Mathematics Measure Theory course: A nice introduction to measure theory and why to study it. My way of learning is to understand first why we must study something. This course covers all the whys well.
• Advanced probability course by Professor Lanchier: An introduction to measure-theoretic probability with some simple examples and some of the essential proofs. Easy to follow and did not scare an engineer.
• Measure Theory by Claudio Landim: A more detailed introduction to measure theory.

The basic and the more advanced things I was studying would sometimes reference ideas from functional analysis, which I never had a course on. Looking back, I would first take a course in functional analysis and then study measure theory. While the notations/way of doing this is different, everything made sense in measure theory after studying functional analysis as well.

• The Bright Side of Mathematics Functional Analysis couse
• 18.102 MIT - Still working on it, a fantastic course.

Now, we can wander into ways of applying these new tools. Some courses I have watched and enjoyed:

• Markov Processes by J.N Corcoran: Easy to follow, introducing a lot of the fundamental tools in applied probability. Professor Corcoran is a great teacher!
• Mathematical Statistics by J.N. Corcoran: Easy to follow, introduces estimators, confidence intervals, binary hypothesis, etc. The major downside of this course is that some courses need to be added.
• High dimensional probability by R. Vershynin: Great course to understand plenty of tools in high dimensional probability. Maybe too much was covered, but it was fascinating, nonetheless.
• Learning Theory by S.B. David: For learning theory people this course needs no introduction. Great introduction to theoretical machine learning.
• Large Scale Machine Learning by R. Salakhutdinov: Another great course in theoretical machine learning.

I never had an introduction to information theory in my undergrad either, yet some notions from IT emerged in machine learning and differential privacy. I followed this course and enjoyed it:

• Information Theory, pattern recognition and neural networks by D. MacKay

In the end, I will add the courses that study some of the things I am interested in:

• Differential Privacy by G. Kamath: One of a kind introduction in differential privacy!
• Sketching Algorithms by J. Nelson: Highly didactic introduction in sketching techniques.

I found out after starting my PhD that a significant component of the PhD life is writing. Here are some resources that I found helpful in writing:

• The Elements of Style
• How to write a great paper by S.P Jones

I am a big fan of Oded Goldreich. His books and blog posts are a mine of gold, yet I will only highlight two of them:

• How to write a paper by O Goldreich
• On the emotional difficulty of conducting research by O. Goldreich