Override some training parameters in train call

[AndreaCossu]

It could be useful to provide to the user the possibility to specify some training parameters like train_epochs or train_mb_size in the train (and/or train_exp) call. In this way, it would be possible to specify different parameters for different experiences. For example, the first experience may contain a larger number of patterns, thus requiring more epochs or larger minibatch sizes.

These additional parameters should override the ones passed at strategy object creation, so that if the user does not pass anything, default ones are used.

What do you think?

[vlomonaco]

Yes, it would be nice to have this feature. Should be also easy to implement via the propagation of optional parameters we have already.

[EdenBelouadah]

I think it's a really interesting feature, and this is exactly what I need for my experiments :)

[AntonioCarta]

I think the easiest way is to override the base class or use a plugin. For example:

class SetSIWHyperparams(StrategyPlugin):
    def __init__(self):
        super().__init__()

    def before_training_exp(self, strategy: 'BaseStrategy', **kwargs):
        if self.experience.current_experience == 0:  # first experience
            self.train_batch_size = 32  # set batch size
            self.optimizer = ...
        else self.experience.current_experience == 0:  # incremental update
            self.train_batch_size = 64  # set batch size
            self.optimizer = ...

    def make_train_dataloader(..):
        pass  # change dataloader here

Probably we can add an example/how-to guide to show this explicitly for the most common cases.

[EdenBelouadah]

@vlomonaco, it's true, for fast prototyping and more intuitive solutions, adding some parameters to the train() method is a good idea, while plugins could be used as a more flexible solution for complicated manipulations.

[EdenBelouadah]

@AntonioCarta . I need the lr_scheduler in the first experience, but with different parameters than in the incremental states. This is the code now, can you please confirm for me that it's correct? Thank you

from torch.optim.lr_scheduler import ReduceLROnPlateau

class HyperParamsPlugin(StrategyPlugin):
    def __init__(self, lr_scheduler, inc_exp_epochs, inc_exp_patience, first_exp_lr,
                 lr_decay):
        super().__init__()
        self.lr_scheduler = lr_scheduler
        self.inc_exp_epochs = inc_exp_epochs
        self.inc_exp_patience = inc_exp_patience
        self.first_exp_lr = first_exp_lr
        self.lr_decay = lr_decay

    def before_training_exp(self, strategy: 'BaseStrategy', **kwargs):
        if strategy.experience.current_experience > 0:  # incremental update
            strategy.train_epochs = self.inc_exp_epochs
            strategy.optimizer.param_groups[0]['lr'] = \
                self.first_exp_lr / strategy.experience.current_experience
            strategy.scheduler = ReduceLROnPlateau(strategy.optimizer,
                                                   patience=self.inc_exp_patience,
                                                   factor=self.lr_decay)

    def after_training_epoch(self, strategy: 'BaseStrategy', **kwargs):
        self.lr_scheduler.step(strategy.loss.cpu().data.numpy())
        lr = strategy.optimizer.param_groups[0]['lr']
        print(f"\nlr = {lr}")
        
 
scheduler = ReduceLROnPlateau(optimizer,
                          patience=args.first_exp_patience,
                          factor=args.lr_decay)

hyperparams = HyperParamsPlugin(scheduler,
                                                   args.inc_exp_epochs,
                                                   args.inc_exp_patience,
                                                   args.first_exp_lr,
                                                   args.lr_decay)

strategy = Naive(model, optimizer, criterion,
                     device=device, train_epochs=args.first_exp_epochs,
                     evaluator=eval_plugin,
                     plugins=[hyperparams],
                     train_mb_size=args.train_batch_size,
                     eval_mb_size=args.eval_batch_size)       

[AntonioCarta]

@EdenBelouadah it seems correct to me. Please check that the printed learning rates show the expected values.

Yes, a plugin is a very flexible way to do this. Still, I think giving the possibility to change basic parameters such as n_epochs, etc directly from the train/eval methods would be nice to have for fast prototyping!

I don't get this point. We are talking about a couple lines of code. Isn't it fast enough?

IMO the problem with adding arguments is that we need lots of them and we will never cover all the possible usecases. Instead, customizing the strategy with a plugin is much more general. For example, the lr scheduler is not used by the BaseStrategy so you still need a plugin for that. The same argument applies if you want to change the dataloader. Should we expose all the dataloader arguments to the user? Should we provide a tutorial for each possible customization?

If implementing a plugin for this usecase is too difficult, maybe this is the problem that we need to address.

[andrearosasco]

Hey there! Just found this discussion and wanted to share my SchedulerPlugin implementation:

class SchedulerPlugin(StrategyPlugin):
    def __init__(self, scheduler):
        super().__init__()
        self.scheduler = scheduler

    def before_training_exp(self, strategy: 'BaseStrategy', **kwargs):
        self.base_lrs = [group['lr'] for group in strategy.optimizer.param_groups]

    def after_training_epoch(self, strategy: 'BaseStrategy', **kwargs):
        self.scheduler.step()

    def after_training_exp(self, strategy: 'BaseStrategy', **kwargs):
        for group, lr in zip(strategy.optimizer.param_groups, self.base_lrs):
            group['lr'] = lr
        self.scheduler._step_count = 0

I wrote this to work with MultiStepLR scheduler.
I guess it would be nice to have something like this added as a utility to Avalanche but it should be general enough to work with different schedulers.

As an alternative it may be useful to feature an example of it in the avalanche documentation/tutorials. I'm afraid that newcomers might just pass the scheduler to their main strategy plugin without fully exploiting the flexibility of plugins.

[AntonioCarta]

If you want to prepare a PR I think we can merge it. There is a difference between mine and your implementation. You reset the lr for each experience. Can we control this behavior using a constructor argument?

If you add this and a quick test to check the lr is updated correctly we can integrate it.

As an alternative it may be useful to feature an example of it in the avalanche documentation/tutorials. I'm afraid that newcomers might just pass the scheduler to their main strategy plugin without fully exploiting the flexibility of plugins.

Yes, I will probably add it as an example.