Feeding single experiences at a time

[highclef]

Hello,

I am currently developing a continual learning framework for my project.
My LSTM is getting amount of batch dataset at different time points (t1, t2, ...), which can be defined as experiences in this case.
However, since each batch dataset is too big, I am considering of saving network parameters of my LSTM (weights, biases) after feeding each batch dataset at t1 as a single experience to the CL strategy. So the sequence is as follows:

1. Train using a single experience (t1)
2. Save network parameters
3. Load network parameters
4. Train using a single experience (t2)
5. Save network parameters
6. ...

And here are my questions:

1. Is there any difference across strategies (Naive, EWC, ...) when I just feed a single experience using tensors_scenario?
2. What is the difference between Avalanche finetune (Naive) and typical PyTorch training method (forward pass, backward & optimization)? I have already seen this:

train
    before_training

    before_train_dataset_adaptation
    train_dataset_adaptation
    after_train_dataset_adaptation
    make_train_dataloader
    model_adaptation
    make_optimizer
    before_training_exp  # for each exp
        before_training_epoch  # for each epoch
            before_training_iteration  # for each iteration
                before_forward
                after_forward
                before_backward
                after_backward
            after_training_iteration
            before_update
            after_update
        after_training_epoch
    after_training_exp
    after_training

but unfortunately it does not give me any idea why it actually showed better performance than the typical PyTorch training method. What exactly does Avalanche Naive method with continual learning concepts?

[AndreaCossu]

Hi @highclef !

1. Avalanche strategies should work regardless of the type of benchmark you use. So, they should work fine in your case.
2. Avalanche Naive strategy simply trains the model on the data continuously, without applying any CL strategies. The optimizer state is reset at the end of each experience. You can open an issue with your code and results if you see significant performance difference.

[highclef]

Hi @AndreaCossu , thank you for the reply and sorry for the late response.

I'll put my codes here so that you can look into it.
Here is the training function of my LSTM based on PyTorch:

def train(model, X_train, y_train, criterion, optimizer, num_epochs):

  training_losses = []

  for epoch in range(1,num_epochs+1):
 
    for i, (input) in enumerate(X_train, 0):
      
      output = y_train[i]
      #Forward pass
      output_model = model(input)
      loss = criterion(output_model, output.view(len(output_model), 6)) 
      training_losses.append(loss.item())

      #Backward and optimize
      optimizer.zero_grad()
      loss.backward()
      optimizer.step()
 
    print("Epoch: %d, train_loss: %1.6f" % (epoch, loss.item()))   
          
  return model 

And here is the comparison of MSELoss values of my training method and the Naive method from Avalanche:

(I recommend you to open this image in new tab when your GitHub screen theme is black.)

So now the discussion points are:

1. Why do I have this huge performance difference? What in Avalanche Naive method does make the better performance in comparison to my training code above?
2. According to the Avalanche documentation, Naive method is a finetuning. However, I was not able to find any code lines that indicates that. Which part is it?

[AndreaCossu]

If you look at the BaseStrategy code you can see that the training part is identical to the one you wrote here in PyTorch. Naive inherits the training loop of BaseStrategy. The only difference may be in the criterion used to compute the loss, since I see you are using a custom criterion. Remember to override the criterion method in Naive and implement your own. Apart from that, performances should be the same on average. If your problem is a classification with 10 classes, the performance is already comparable (slightly lower for the avalanche implementation but you should execute multiple runs to ensure that this difference is relevant, when at random performance results may vary).

[highclef]

I think there are some points of missunderstanding, sorry that I have forgotten to mention:

1. I didn't used a custom criterion. I only used MSELoss() from PyTorch (criterion = nn.MSELoss()), so the both criterions should be the same for Naive and my training method.
2. Naive showed actually better performance than my training method. You can see that on the graph above: the lower the value, the better the performance (since those are loss values).
3. I am using these strategies on regression problem.
4. I already executed the code multiple times, but every time it showed amount of difference between Naive and my training method, so I think there should be something inside in Naive that makes this happen.
5. The second question from my previous post has not been answered yet. Why is Naive a finetuning?