This blog will record the timeline, resources, projects along the way of my study on machine learning since 2015. And I will keep updating this blog.

### Mathematics

- Basics: better to review before real study of ML
- Calculus
- Linear Algebra
- Introduction to Linear Algebra: a nice textbook in linear algebra (
**First Pass**in*Dec, 2015*). - MIT 18.06: Linear algebra (
**Completed**in*Dec, 2015*).

- Introduction to Linear Algebra: a nice textbook in linear algebra (
- Probability Theory

- Reference
- Handbook of Mathematics: an amazing reference book of mathematics.
- Problem-Solving Through Problems: Polish up your math skills.

- Statistics
- All of Statistics: A textbook appealing to MLers (
**First Pass**in*Aug, 2017*). - Statistical Inference: Classic textbook.
- CMU 10-705: Intermediate Statistics (
**Completed**in*Aug, 2017*). - Post Series: Doubt Clarification for Statistics (In
*Chinese*)

- All of Statistics: A textbook appealing to MLers (
- Optimization
- Convex Optimization: Classic textbook. (
**First Pass**in*Nov, 2018*) - Stanford EE364a: Convex Optimization. (
**Completed**in*Nov, 2018*)

- Convex Optimization: Classic textbook. (
- Advanced Topics on Linear Algebra
- The Matrix Cookbook: Ongoing
- Matrix Analysis: Ongoing

- Advanced Topics on Statistics

### Statistical Machine Learning

- Coursera: Machine Learning: Introduction to machine learning. (Certificate)
- StanfordOnline: Statistical Learning: Introduction to statistical learning and
**R**. (Certificate) - UAlberta CMPUT 466/551: Machine Learning. I’m one of TAs in this course.
- Post Series: ML with R

- The Elements of Statistical Learning: The first six chapters are highly recommended, while some later chapters are a little bit out-of-dated from my perspective (
**First Pass**in*Feb, 2018*).- Manual: Figure out the mathematical details.
- R package “ElemStatLearn”: Contain data sets, functions and examples from the book.
- My code: Exercises and experiment reimplementations in
**R**. - Post Series: Notes on Mathematics for ESL

- Machine Learning: A Probabilistic Perspective: In the view of Bayesian to interpret ML. (
**First Pass**in*Jan, 2019*).

### Probabilistic Graphical Models:

- UAlberta CMPUT 659: Probabilistic Graphical Models. Graduate course in University of Alberta based on Coursera course instructed by Koller (Report: A Topic Model of Genetic Mutations in Cancer,
**Completed**in*Apr, 2016*). - CMU 10-708: Probabilistic Graphical Models.
- Probabilistic Graphical Models: Principles and Techniques: Classic textbook for probabilistic graphical models. I’ve read a few chapters of it and gave up. This book is somewhat too mathematical and I will not recommend this book for beginners in PGM.
- Pomegranate: A Python package for PGM. Nice tutorials for beginners in PGM!

### Deep Learning

- Neural Network and Deep Learning: Nice online tutorial for beginners in deep learning (
**Completed**in*Sep, 2016*). - Coursera: Neural Networks for Machine Learning: Excellent course for beginners in neural networks. (Certificate)
- Stanford CS224d: Deep Learning for Natural Language Processing (
**Completed**in*Apr, 2017*). - Stanford CS231n: Convolutional Neural Networks for Visual Recognition.
- Stanford CS236: Deep Generative Models. (
**Completed**in*Dec, 2018*) - Deep Learning Book: First and second part are great to read and need to be understood well. (
**First Pass**in*Oct, 2018*) - TensorFlow: Large amount of high-quality tutorials and well-coded recent advances in DL. Most applicable scenario: research.
- PyTorch: Use dynamic graph instead of static graph in TensorFlow. Relatively easy to use compared to TensorFlow. Most applicable scenario: research.
- Mxnet: High performance. Most applicable scenario: industry.
- Keras: High-level, easy to use. Most applicable scenario: competitions like Kaggle.

Note: I’m comfortable with coding in both TensorFlow and PyTorch and use Keras from time to time. I don’t know much about Mxnet and rarely use it, but I’m amazed by their group members.

### Large Scale Machine Learning

- edX: Introduction to Big Data with Apache Spark: Introduction to Spark in
**Python**. (Certificate) - edX: Scalable Machine Learning: Explore deeper usage of Spark on large scale machine learning. (Certificate)
- Coursera: Functional Programming Principles in Scala: Introduction to
**Scala**. (Certificate) - Coursera: Functional Program Design in Scala: Further study on
**Scala**. (Certificate) - Coursera: Parallel Programming: Parallel computing in
**Scala**. (Certificate) - Coursera: Big Data Analysis with Scala and Spark: Intro to
**Spark**with**Scala**. (Certificate) - Coursera: Big Data Analysis with Scala and Spark: Intro to
**Spark**with**Scala**. (Certificate) - Coursera: Functional Programming in Scala Capstone: Capstone project for
**Scala**. (Certificate)

### Reinforcement Learning

- UAlberta CMPUT 609: Reinforcement Learning. Graduate course in UofA instructed by Sutton. (Report: Investigation on Deep Reinforcement Learning Methods for Classical Control Problems,
**Completed**in*Dec, 2016*). - Introduction to Reinforcement Learning: Classic textbook, recommended for beginners in RL (
**First Pass**in*Dec, 2016*). - RL Course from DeepMind: Instructed by David Silver. Highly recommended for beginners in RL (
**Completed**in*Oct, 2018*) - Bandit Algorithms Blog: A great blog discussing all kinds of bandit algorithms by Csaba (
**First Rough Pass**in*Feb, 2018*). - Post Series: Notes on Reinforcement Learning

### Related Topics

- Information Theory
- Data Mining
- Coursera: Mining of Massive Datasets: Introduction to a lot data mining techniques. (Certificate)

- Game Theory
- Coursera: Game Theory: Introduction to game theory. (Certificate)

### ML Competitions

- Kaggle: Home Depot Product Search Relevance: Top 10%. (Report)
- ATEC, NLP for Financial Integellience (hosted by Alibaba): 10th out of 443 teams (Certificate)