Permute MNIST by row

[kushalj001]

The current implementation of PermutedMNIST permutes all the pixels by default. That is, it finds a permutation after flattening the image into a vector of dim 784. I am trying to train an RNN model that takes in images by rows. That is my input_dim = 28 and seq_len=28. This is similar to the row LSTM architecture proposed in this paper. For this, I need to permute the images by their row rather than by pixels. Is there any way to do this using the Avalanche classic benchmarks currently?
Thank you

[vlomonaco]

Hi @kushalj001!

At the moment the classic "PermutedMNIST" does not offer this feature but this should be rather easy to implement with a custom transformation! Here's the custom transformation used for the classic permutation. You can have a similar approach for your variation. Of course feel free to open a PR if you want to contribute back to Avalanche, that would be great and we can give you feedback about the implementation!

[kushalj001]

Thanks for the reply, can you take a brief look at this implementation? I have only changed 2 lines. In the PermutedMNIST method, instead of creating a random permutation of 784, I create a permutation of 28. (This can be dynamic by taking in an additional parameter in the function like sequence_length or pixel_per_input, which by default could be 1).

def PermutedMNIST(
    n_experiences: int,
    *,
    seed: Optional[int] = None,
    train_transform: Optional[Any] = _default_mnist_train_transform,
    eval_transform: Optional[Any] = _default_mnist_eval_transform,
    dataset_root: Union[str, Path] = None
) -> NCScenario:
  
    list_train_dataset = []
    list_test_dataset = []
    rng_permute = np.random.RandomState(seed)

    mnist_train, mnist_test = _get_mnist_dataset(dataset_root)

    # for every incremental experience
    for _ in range(n_experiences):
        # choose a random permutation of the pixels in the image
        idx_permute = torch.from_numpy(rng_permute.permutation(28)).type(
            torch.int64
        )

        permutation = PixelsPermutation(idx_permute)

        permutation_transforms = dict(
            train=(permutation, None), eval=(permutation, None)
        )

        # Freeze the permutation
        permuted_train = AvalancheDataset(
            mnist_train,
            transform_groups=permutation_transforms,
            initial_transform_group="train",
        ).freeze_transforms()

        permuted_test = AvalancheDataset(
            mnist_test,
            transform_groups=permutation_transforms,
            initial_transform_group="eval",
        ).freeze_transforms()

        list_train_dataset.append(permuted_train)
        list_test_dataset.append(permuted_test)

    return nc_benchmark(
        list_train_dataset,
        list_test_dataset,
        n_experiences=len(list_train_dataset),
        task_labels=True,
        shuffle=False,
        class_ids_from_zero_in_each_exp=True,
        one_dataset_per_exp=True,
        train_transform=train_transform,
        eval_transform=eval_transform,
    )

The next method is the PixelsPermutation as you mentioned in your answer. Here to implement the transformation, I have just changed the original line to img[:, self.index_permutation] where img has dimensions [1,28,28] and self.index_permutation is [28]. Are there any more changes that I need to make?

class PixelsPermutation(object):
    """
    Apply a fixed permutation to the pixels of the given image.

    Works with both Tensors and PIL images. Returns an object of the same type
    of the input element.
    """

    def __init__(self, index_permutation: Sequence[int]):
        self.permutation = index_permutation
        self._to_tensor = ToTensor()
        self._to_image = ToPILImage()

    def __call__(self, img: Union[Image, Tensor]):
        is_image = isinstance(img, Image)
        if (not is_image) and (not isinstance(img, Tensor)):
            raise ValueError("Invalid input: must be a PIL image or a Tensor")

        if is_image:
            img = self._to_tensor(img)

        img = img[:, self.permutation]

        if is_image:
            img = self._to_image(img)

        return img

Thank you

[AndreaCossu]

Hi @kushalj001 , the changes you described should suffice. The best way to check is always to plot some of the examples 😄