You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If old_location has not been specified, then relative_path should be required.
If old_location has not been specified, it's not necessarily an error for relative_path to match the value for another, already-registered dataset. This actually can happen for GCRCatalogs datasets. The code should perhaps put out a warning but should go ahead and register the new dataset.
The text was updated successfully, but these errors were encountered:
If old_location has not been specified, then relative_path should be required.
I have added this check in the PR (if location_type=dataregistry, as all other types dont handle data)
If old_location has not been specified, it's not necessarily an error for relative_path to match the value for another, already-registered dataset. This actually can happen for GCRCatalogs datasets. The code should perhaps put out a warning but should go ahead and register the new dataset.
It only raises this error when location_type=dataregistry, to make sure people can't over write other peoples data. I'm not sure what you mean by GCR datasets in this sense
When registering a GCRCatalog-type catalog, the code gets as much information as it can from its config file. There are various keywords in those files which can be used to say "the dataset is in this directory". But there are also ways in the config file to narrow down exactly which files constitute the dataset. See for example https://github.com/LSSTDESC/gcr-catalogs/blob/master/GCRCatalogs/catalog_configs/dc2_redmagic_run2.2i_dr6_wfd_v0.8.1_highdens.yaml and https://github.com/LSSTDESC/gcr-catalogs/blob/master/GCRCatalogs/catalog_configs/dc2_redmagic_run2.2i_dr6_wfd_v0.8.1_highlum.yaml
They have the same value for the keyword catalog_root_dir, which is essentially our relative_path, but there is another keyword catalog_path_template which can be used to select a subset of the files in the directory. I thought of three different ways we could handle this, but the simplest by far is just to say that, when dataregistry is not copying files (so there is no danger of overwriting anything), we assume the user knows what they're doing and allow them to use the same relative_path.
old_location
has not been specified, thenrelative_path
should be required.old_location
has not been specified, it's not necessarily an error forrelative_path
to match the value for another, already-registered dataset. This actually can happen for GCRCatalogs datasets. The code should perhaps put out a warning but should go ahead and register the new dataset.The text was updated successfully, but these errors were encountered: