From Zero to Research Scientist full resources guide.

Guide description

This guide is designated to anybody with basic programming knowledge or a computer science background interested in becoming a Research Scientist with 🎯 on Deep Learning and NLP.

You can go Bottom-Up or Top-Down both works well and it is actually crucial to know which approach suites you the best. If you are okay with studying lots of mathematical concepts without application then use Bottom-Up. If you want to go hands-on first then use the Top-Down first.

Contents:

• Mathematical Foundation
  • Linear Algebra
  • Probability
  • Calculus
  • Optimization Theory
• Machine Learning
• Deep Learning
• Reinforcement Learning
• Natural Language Processing

Mathematical Foundations:

The Mathematical Foundation part is for all Artificial Intelligence branches such as Machine Learning, Reinforcement Learning, Computer Vision and so on. AI is heavily math-theory based so a solid foundation is essential.

Linear Algebra

♾️

This branch of Math is crucial for understanding the mechanism of Neural Networks which are the norm for NLP methodologies in nowadays State-of-The-Art.

Resource	Difficulty	Relevance
MIT Gilbert Strang 2005 Linear Algebra 🎥	★★☆☆☆
Linear Algebra 4th Edition by Friedberg 📘	★★★★☆
Mathematics for Machine Learning Book: Chapter 2 📘	★★★☆☆
James Hamblin Awesome Lecture Series 🎥	★★★☆☆
3Blue1Brown Essence of Linear Algebra 🎥	★☆☆☆☆
Mathematics For Machine Learning Specialization: Linear Algebra 🎥	★☆☆☆☆
Matrix Methods for Linear Algebra for Gilber Strang UPDATED! 🎥	★★★☆☆

Probability

Most of Natural Language Processing and Machine Learning Algorithms are based on Probability theory. So this branch is extremely important for grasping how old methods work.

Resource	Difficulty	Relevance
Joe Blitzstein Harvard Probability and Statistics Course 🎥	★★★★★
MIT Probability Course 2011 Lecture videos 🎥	★★★☆☆
MIT Probability Course 2018 short videos UPDATED! 🎥	★★☆☆☆
Mathematics for Machine Learning Book: Chapter 6 📘	★★★☆☆
Probalistic Graphical Models CMU Advanced 🎥	★★★★★
Probalistic Graphical Models Stanford Daphne Advanced 🎥	★★★★★
A First Course In Probability Book by Ross 📘	★★★★☆
Joe Blitzstein Harvard Professor Probability Awesome Book 📘	★★★☆☆

Calculus

📐

Resource	Difficulty	Relevance
Essence of Calculus by 3Blue1Brown🎥	★★☆☆☆
Single Variable Calculus MIT 2007🎥	★★★★☆
Strang's Overview of Calculus🎥	★★★★☆
MultiVariable Calculus MIT 2007🎥	★★★★★
Princeton University Multivariable Calculus 2013🎥	★★★★☆
Calculus Book by Stewart 📘	★★★★☆
Mathematics for Machine Learning Book: Chapter 5 📘	★★★☆☆

Optimization Theory

📉

-Resource	Difficulty	Relevance
CMU optimization course 2018🎥	★★★★★
CMU Advanced optimization course🎥	★★★★★
Stanford Famous optimization course 🎥	★★★★★
Boyd Convex Optimization Book 📕	★★★★★

Machine Learning

Considered a fancy name for Statistical models where its main goal is to learn from data for several usages. It is considered highly recommended to master these statistical techniques before Research as most of research is inspired by most of the Algorithms.

Resource	Difficulty Level
Mathematics for Machine Learning Part 2 📚
Pattern Recognition and Machine Leanring📚
Elements of Statistical Learning 📚
Introduction to Statistical Learning 📚
Machine Learning: A Probalisitic Perspective 📚
Berkley CS188 Introduction to AI course 🎥
MIT Classic AI course taught by Prof. Patrick H. Winston 🎥
Stanford AI course 2018 🎥
California Instuite of Technology Learning from Data course 🎥
CMU Machine Learning 2015 10-601 🎥
CMU Statistical Machine Learning 10-702 🎥
Information Theory, Pattern Recognition ML course 2012 🎥
Large Scale Machine Learning Toronto University 2015 🎥
Algorithmic Aspects of Machine Learning MIT 🎥
MIT Course 9.520 - Statistical Learning Theory and Applications, Fall 2015 🎥
Undergraduate Machine Learning Course University of British Columbia 2013 🎥

Deep Learning

One of the major breakthroughs in the field of intersection between Artificial Intelligence and Computer Science. It lead to countless advances in technology and considered the standard way to do Artificial Intelligence.

Resource	Difficulty Level
Deep Learning Book by Ian Goodfellow 📚
UCL Deepmind Deep Learning 🎥
Advanced Talks by Deep Learning Pioneers 🎥
Stanford Autumn 2018 Deep Learning Lectures 🎥
FAU Deep Learning 2020 Series 🎥
CMU Deep Learning course 2020 🎥
Stanford Convolutional Neural Network 2017 🎥
Oxford Deep Learning Awesome Lectures 2015 🎥
Stanford NLP with Deep Learning 2019 🎥
Deep Learning from Probability and Statistics POV 🎥
Advanced Deep Learning UCL 2017 course + Reinforcement Learning 🎥
Deep Learning UC Berkley 2020 Course 🎥
NYU Deep Learning with Pytorch hands on 🎥
Classic Jeoffrey Hinton Old course OUTDATED 🎥
Pieter Abdeel Deep Unsupervised Learning 🎥
Hugo Larochelle Deep Learning series 🎥
Deep Learning Book Explanation Series 🎥
Deep Learning Introduction by Durham University 🎥
Fast.ai Practical Deep Learning 🎥
Fast.ai Deep Learning From Foundations 🎥
Deep Learning with Python (Keras Author) 📚

Reinforcement Learning

It is a sub-field of AI which focuses on learning by observation/rewards.

Resource	Difficulty Level
Introduction to Reinforcement Learning 📚
David Silver Deep Mind Introductory Lectures 🎥
Stanford 2018 cs234 Reinforcement Learning🎥
Stanford 2019 cs330 Meta Learning advanced course 🎥
Sergie Levine 2018 UC Berkley Lecture Videos 🎥
Waterloo cs885 Reinforcement Learing 🎥
Sergie Levine 2020 Deep Reinforcement Learning 🎥
Reinforcement Learning Specialization Coursea GOLDEN courses🎥 (Though it is not free but you can apply for financial aid)

Natural Language Processing

It is a sub-field of AI which focuses on the interpretation of Human Language.

Resource	Difficulty Level
Jurafsky Speech and Language Processing 📚
Christopher Manning Foundations of Statistical NLP📚
Christopher Manning Introduction to Information Retrieval📚
cs224n Natural Language Processing with Deep Learning GOLDEN 2019🎥
Oxford Natural Language Processing with Deep Learning 2017🎥
Michigan Introduction to NLP🎥
cs224u Natural Language Understanding 2019 🎥
cmu 2021 Neural Nets for NLP 2021🎥
Jurafsky and Manning Introduction to Natural Language Processing🎥

Must Read NLP Papers:

In this section, I am going to list the most influential papers that help people who want to dig deeper into the research world of NLP to catch up.

TODO