Codes and Projects for Machine Learning Course, University of Tabriz.
Chapter 1: Introduction (video)
- download slides in Persian (pdf)
- Linear regression
- Gradient descent algorithm (video)
- Multi-variable linear regression
- Polynomial regression (video)
- Normal equation
- Locally weighted regression
- Probabilistic interpretation (video)
- Download slides in Persian (pdf)
- Python basics
- Creating vectors and matrices in
numpy - Reading and writing data from/to files
- Matrix operations (video)
- Colon (:) operator
- Plotting using
matplotlib(video) - Control structures in python
- Implementing linear regression cost function (video)
Chapter 4: Logistic Regression (video)
- Classification and logistic regression
- Probabilistic interpretation
- Logistic regression cost function
- Logistic regression and gradient descent
- Multi-class logistic regression
- Advanced optimization methods
- Download slides in Persian (pdf)
- Artificial Intelligence: A Modern Approach (3rd Edition), pages 725-727
- An Introduction to Statistical Learning: with Applications in R, pages 130-137
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction, pages 119-128
Chapter 5: Regularization (video)
- Overfitting and Regularization
- L2-Regularization (Ridge)
- L1-Regularization (Lasso)
- Regression with regularization
- Classification with regularization
- Download slides in Persian (pdf)
Chapter 6: Neural Networks (video)
- Milti-class logistic regression
- Softmax classifier
- Training softmax classifier
- Geometric interpretation
- Non-linear classification
- Neural Networks (video: part 2)
- Training neural networks: Backpropagation
- Training neural networks: advanced optimization methods (video: part 3)
- Gradient checking
- Mini-batch gradient descent
- Download slides in Persian (pdf)
- Step by step Implementation of a multi-layer neural network in Python
- Backpropagation algorithm: Step by step example
- Activation functions (Sigmoid, Tanh, ReLU, PReLU, Maxout) and weight initialization methods
- How to solve problems using neural networks
- Dropout: A Simple Way to Prevent Neural Networks from Overfitting
- Neurons and how they work
- Neural Networks and Deep Learning; Michael Nielsen: This book is a very good place to start learning about neural networks and deep learning.
- Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville: For a more technical review of neural networks and deep learning, I recommend this book.
- Motivation: optimal decision boundary (video: part 1)
- Support vectors and margin
- Objective function formulation: primal and dual
- Non-linear classification: soft margin (video: part 2)
- Non-linear classification: kernel trick
- Multi-class SVM
- Download slides in persian (pdf)
Chapter 8: Clustering (video)
- Supervised vs unsupervised learning
- Clustering
- K-Means clustering algorithm (demo)
- Determining number of clusters: Elbow method
- Postprocessing methods: Merge and Split clusters
- Bisectioning clustering
- Hierarchical clustering
- Application 1: Clustering digits
- Application 2: Image Compression
- Download slides in Persian (pdf)
Chapter 9: Dimensionality Reduction and PCA (video)
- Introduction to PCA
- PCA implementation in python
- PCA Applications
- Singular Value Decomposition (SVD)
- Downloas slides in Persian (pdf)
- Intoduction to anomaly detection
- Some applications (security, manufacturing, fraud detection)
- Anoamly detection using probabilitic modelling
- Uni-variate normal distribution for anomaly detection
- Multi-variate normal distribution for anomaly detection
- Evaluation measures (TP, FP, TN, FN, Precision, Recall, F-score)
- Anomaly detection as one-class classification
- Classification vs anomaly detection
- Download slides in Persian (pdf)
Chapter 11: Recommender Systems (video)
- Introduction to recommender systems
- Collaborative filtering approach
- User-based collaborative filtering
- Item-based collaborative filtering
- Similarity measures (Pearson, Cosine, Euclidian)
- Cold start problem
- Singular value decomposition
- Content-based recommendation
- Cost function and minimization
- Download slides in Persian (pdf)
- Optimization: Convex Optimization, Stephan Boyd, Stanford
- Linear algebra: pdf
- Calculus: Khan Accademy
- Probability: Khan Accademy
- Regression and Gradient Descent
- Classification, Logistic Regression and Regularization
- Multi-Class Logistic Regression
- Neural Networks Training
- Neural Networks Implementing
- Clustering
- Dimensionallity Reduction and PCA
- Recommender Systems