Reuse dataset transformations for replay

[AntonioCarta]

Is it possible to reuse dataset transformations for replay buffers? How should we do it?

The problem is that transform include preprocessing + augmentation, but after you save them in a buffer you only need to apply augmentation. Applying the preprocessing again is a mistake.

See #637 for more discussion.

@lrzpellegrini

[lrzpellegrini]

This is a nice but yet tricky feature to implement. The problem with augmentations is that we don't have a systematic way to detect if a transformation (basically a callable) is an augmentation (with "randomness") or a deterministic preprocessing step.

In fact, an empirical evaluation can be done by "trying" each transformation twice to check if we get a different output for the same input.

In general, I think it's a good idea to have some methods to enable/disable augmentations. This may also open to the possibility to have some kind of in-memory caching of the dataset.

However, consider that this would only discriminate between random and deterministic transformations. The semantic difference between "preprocessing" and "augmentation" may be harder to define...

[AntonioCarta]

I agree with you that it's tricky. I am not convinced that determinism is a sufficient condition and it can be hard to find bugs if we rely on this.

I think leaving this choice to the user is fine. We can provide a transform argument to replay methods instead of extracting transformations from datasets.