Minor updates and bug fixes

Minor updates

• Update functions for object detection models (PR #59)
• Update README (PRs #61 #62)

Minor bug fixes

• Rename (PR #60)
• Bug fixes (PR #73)

Support more detailed training configs and update official configs

Updated official README and configs

• More detailed instructions (PRs #55, #56)
• Restructured official configs (PR #55)
• Updated FT config for ImageNet (PR #55)

Support detailed training configurations

• Step-wise parameter update besides epoch-wise parameter update (PR #58)
• Gradient accumulation (PR #58)
• Max gradient norm (PR #58)

Bug/Typo fixes

• Bug fixes (PRs #54, #57)
• Typo fixes (PRs #53, #58)

Google Colab Examples and bug fixes

New examples

• Added sample configs for CIFAR-10 and CIFAR-100 datasets

1. Training without teacher (i.e., using TrainingBox) for CIFAR-10 and CIFAR-100 (PR #48)
2. Knowledge distillation for CIFAR-10 and CIFAR-100 (PR #50)

• Added Google Colab examples (PR #51)

1. Training without teacher for CIFAR-10 and CIFAR-100
2. Knowledge distillation for CIFAR-10 and CIFAR-100

Bug fixes

• Fixed a bug in init of DenseNet-BC (PR #48)
• Resolved checkpoint name conflicts (PR #49)

TrainingBox, PyTorch Hub, random split, pretrained models for CIFAR-10 and CIFAR-100 datasets

New features

• Added TrainingBox to train models without teachers (PR #39)
• Supported PyTorch Hub in registry (PR #40)
• Supported random split e.g., split training dataset into training and validation datasets (PR #41)
• Added reimplemented models for CIFAR-10 and CIFAR-100 datasets (PR #41)

Pretrained models

Referred to the following repositories for training methods.

• ResNet: https://github.com/facebookarchive/fb.resnet.torch
• WRN (Wide ResNet): https://github.com/szagoruyko/wide-residual-networks
• DenseNet-BC: https://github.com/liuzhuang13/DenseNet

Note that there are some accuracy gaps between these and those reported in their original studies.

	CIFAR-10	CIFAR-100
ResNet-20	91.92	N/A
ResNet-32	93.03	N/A
ResNet-44	93.20	N/A
ResNet-56	93.57	N/A
ResNet-110	93.50	N/A
WRN-40-4	95.24	79.44
WRN-28-10	95.53	81.27
WRN-16-8	94.76	79.26
DenseNet-BC (k=12, depth=100)	95.53	77.14

Extended ForwardHookManager and bug fix

• Extended ForwardHookManager (Issue #32 PR #33)
• Fixed bugs around post_forward function caused by a gathering paradigm introduced to I/O dict (Issue #34 PR #35)

The first release of torchdistill

torchdistill

The first release of torchdistill with code and assets for "torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation"