Left over from release

[stuartmcalpine]

I've made some minor comments about the tutorials (which you probably can't yet see since I haven't submitted a review) just from looking at the diffs. I intend to also look at the full text of the tutorials at some point but it's not critical that it happen in time for the release. I started looking at the documentation but thought you might still be working on it so I'll hold off until you tell me you're ready.
I found one other small thing in the creation script which I couldn't comment on easily since it's not in code you changed. In _get_column_definitions, line 79, you use the variable prod_schema which has been set by the main part of the script at line 314. Yes, it's available to the routine, but it's not a good practice to use it this way. If we decided to put the helper routines in a separate file it would break. Better to pass it as an argument.

• installation.rst needs an overhaul

• do we want to mention reg_reader and reg_writer in there and expected use of them?

• in tutorial_CLI could we change the example owner from "DESC Generic Working Group" to something without blanks? E.g. text could be "For example to see all the datasets from the DESC Generic Working Group (GWG)..." and then value for owner could be GWG. It should work as it stands, but I'd rather not encourage people to use values with embedded spaces.

• in tutorial_python.rst there is no link to the production_schema tutorial

• Update Postgres version. It appears we need at least 15.2 for NULLS NOT DISTINCT. See https://blog.rustprooflabs.com/2022/07/postgres-15-unique-improvement-with-null

  • Update posgres version Updated to 16.4.alpine
  • Move spin instance to production
  • Update to a persistent volume
  • Add developer notes on basic SPIN setup process

• Decide on new default NERSC site, and update src/dataregistry/site_config/site_rootdir.yaml. Updated to /global/cfs/cdirs/lsst/utilities/data-registry

• Decide on new schema default (SCHEMA_VERSION in src/dataregistry/db_basic.py). Default schema names are now at src/dataregistry/schema/default_schema_names.yaml, one for WORKING and one for PRODUCTION

• Implement tutorial schema for tutorial notebooks and update notebooks to point to it Created with the new schema creation script, tutorial_working and tutorial_production, will update notebooks to match

• Review tutorial notebooks

  • register_datasets.ipynb
  • datasets_deeper_look.ipynb
  • production_schema.ipynb
  • query_datasets.ipynb
  • pipelines.ipynb

• Review documentation

  • installation
  • tutorial
  • reference
  • dev_notes
  • contact

• Create startup script to create schema and owner_type directories. These should be static, and that level of the directory structure should have no write access for regular users. New script in scripts folder called create_schema_dirs.sh that creates the initial schema directories and their owner_type groups but needs some revision.

• Get registry working with communal (read only?) pgpass and config files at NERSC

  • Can have a communal config file at least. Update reading of the config file to start with looking for the NERSC env variable that points to the config file (and update docs to reflect this)

• Get gcrcatalogs working with release

• Exercise sqlite I've tested sqlite locally using the create script and running the CI tests, works good for me

• Build github release and publish to PyPi (have CI publish releases automatically to PyPi?)

  • Update installation to talk about installing from PyPi

• Incorporate registry into python_bleed.

  • Update installation notes to reflect this