Lab 2

Lab Notebooks:

Lab scripts:

See CIFAR10 for a training and inference example on CIFAR-10.

For self-study (all students):

CNN Explainer
But what is a convolution? (convolution example; convolutions in image processing; convolutions and polynomial multiplication; FFT)
https://paperswithcode.com/method/1x1-convolution
TorchVision transforms getting started: https://pytorch.org/vision/main/auto_examples/transforms/plot_transforms_getting_started.html
TorchVision examples: https://pytorch.org/vision/stable/auto_examples/transforms/plot_transforms_illustrations.html#sphx-glr-auto-examples-transforms-plot-transforms-illustrations-py

Advanced (for students who want to learn more):

pin_memory & non_blocking=True:
- Pinning memory in DataLoaders: https://pytorch.org/docs/stable/notes/cuda.html#use-pinned-memory-buffers
- How does pinned memory actually work: https://developer.nvidia.com/blog/how-optimize-data-transfers-cuda-cc/
- Also see this discussion: https://discuss.pytorch.org/t/should-we-set-non-blocking-to-true/38234/4
Data Augmentation Techniques:
- RandAugment: Practical automated data augmentation with a reduced search space
- Regularization Strategy to Train Strong Classifiers with Localizable Features
- mixup: Beyond Empirical Risk Minimization
- RandAugment in torchvision: https://pytorch.org/vision/main/generated/torchvision.transforms.v2.RandAugment.html
- How to use CutMix and MixUp: https://pytorch.org/vision/main/auto_examples/transforms/plot_cutmix_mixup.html

References:

DataLoader: https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader
Dataset: https://pytorch.org/docs/stable/data.html#torch.utils.data.Dataset
Datasets & DataLoaders example: https://pytorch.org/tutorials/beginner/basics/data_tutorial.html
CIFAR-10 and CIFAR-100: https://www.cs.toronto.edu/~kriz/cifar.html
CIFAR-10 training example: https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html

Provide feedback