Skip to content

Commit

Permalink
Fix torch.load() in PersistentDataset and GDSDataset (#8177)
Browse files Browse the repository at this point in the history
### Description
Frequently getting warning massage in a newer Pytorch version (2.4.1 in
my case):
"_You are using `torch.load` with `weights_only=False` (the current
default value), which uses the default pickle module implicitly. It is
possible to construct malicious pickle data which will execute arbitrary
code during unpickling (See
https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models
for more details). In a future release, the default value for
`weights_only` will be flipped to `True`. This limits the functions that
could be executed during unpickling. Arbitrary objects will no longer be
allowed to be loaded via this mode unless they are explicitly
allowlisted by the user via `torch.serialization.add_safe_globals`. We
recommend you start setting `weights_only=True` for any use case where
you don't have full control of the loaded file. Please open an issue on
GitHub for any issues related to this experimental feature._"

This pull request fixes an issue with the `torch.load()` function in the
`PersistentDataset` and `GDSDataset` classes by adding
`weights_only=False`. The fix ensures that the `torch.load()` function
maintains load consistency in future versions of PyTorch.

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: bnbqq8 <[email protected]>
  • Loading branch information
bnbqq8 authored Oct 25, 2024
1 parent 5b2d78e commit 82298ad
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions monai/data/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import warnings
from collections.abc import Callable, Sequence
from copy import copy, deepcopy
from inspect import signature
from multiprocessing.managers import ListProxy
from multiprocessing.pool import ThreadPool
from pathlib import Path
Expand Down Expand Up @@ -371,7 +372,10 @@ def _cachecheck(self, item_transformed):

if hashfile is not None and hashfile.is_file(): # cache hit
try:
return torch.load(hashfile)
if "weights_only" in signature(torch.load).parameters:
return torch.load(hashfile, weights_only=False)
else:
return torch.load(hashfile)
except PermissionError as e:
if sys.platform != "win32":
raise e
Expand Down Expand Up @@ -1670,4 +1674,7 @@ def _load_meta_cache(self, meta_hash_file_name):
if meta_hash_file_name in self._meta_cache:
return self._meta_cache[meta_hash_file_name]
else:
return torch.load(self.cache_dir / meta_hash_file_name)
if "weights_only" in signature(torch.load).parameters:
return torch.load(self.cache_dir / meta_hash_file_name, weights_only=False)
else:
return torch.load(self.cache_dir / meta_hash_file_name)

0 comments on commit 82298ad

Please sign in to comment.