Using Avalanche in Production

[DannyFGitHub]

I want to use Avalanche in a demo for training or evaluating as input comes in.
Is it possible to put Avalanche in a training loop whereby I can send 1 image to train on or evaluate on within that loop?
How do I create a never ending train stream or test stream, whats the best way to do this?

Rough thoughts:
Flask HTTP Request trigger > Process Image > Run through Transformer > Add to Stream > Train if classification is not null.
Flask HTTP Request trigger > Process Image > Run through Transformer > Add to Stream > Evaluate if classification is null and return classification

Pseudo Code:
While True:
     if queue.hasItem():
         image, label = queue.pop()
         image_tensors = transform(image)
         if label !== None:
             metrics = strategy.train(image_tensors, label)
             save_to_db(metrics)
         else:
             classification = strategy.eval(image_tensors)
             save_classification_to_db(classification)

[vlomonaco]

Hi @DannyFGitHub! Thanks for your very interesting question. Unfortunately we officially do not support Avalanche for production settings. So there's no easy way at the moment to manage infinite streams of data or create new experiences on the fly!

In theory you should be able to do this by creating a new benchmark instance every time with only 1 experience and use that for train or eval. Creating an experience directly may be possible as well but you need to know a bit more about the internals of Avalanche.

Let us know if you find a walkaround as we would be interested in supporting this kind of usage of Avalanche in the future! Maybe @lrzpellegrini has some better ideas on this.