Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Adds ability to set alternate local folder for config/metadata directory #109

Closed
wants to merge 3 commits into from

Conversation

cfleur
Copy link
Contributor

@cfleur cfleur commented Oct 4, 2024

WIP!

Closes #108

Based on v1.3.2 due to problems running retrievals on v1.4.0 (see #107).

Config directory is specified in config_setup.yml.

Includes cherry pick of Use sys.executable instead of .venv/bin/python to run retrievals

cfleur and others added 3 commits October 4, 2024 14:14
This way, the retrieval does not fail when running with a Python interpreter that is not a `venv` like virtual environment.

Co-authored-by: Catherine Fait <[email protected]>
Config directory is specified in config_setup.yml at the project root so
that alternate local config/metadata dir can be found
Copy link

netlify bot commented Oct 4, 2024

Deploy Preview for em27-retrieval-pipeline ready!

Name Link
🔨 Latest commit 5db6172
🔍 Latest deploy log https://app.netlify.com/sites/em27-retrieval-pipeline/deploys/66fff7bf8ca39000089eb23e
😎 Deploy Preview https://deploy-preview-109--em27-retrieval-pipeline.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@cfleur
Copy link
Contributor Author

cfleur commented Oct 4, 2024

@dostuffthatmatters do you have any comments on this?
TODO: clean up repeated code, maybe something else

@dostuffthatmatters
Copy link
Member

dostuffthatmatters commented Oct 5, 2024

This is a great idea! Thank you.

We actually just started a second metadata repository because one of our measurement campaigns produced so much metadata.

I would prefer if one passes the alternative config directory via the command line instead of a separate config_setup.yaml file.

What do you think?

@cfleur
Copy link
Contributor Author

cfleur commented Oct 8, 2024

@dostuffthatmatters I tested the fix for pressure null values #107 and its working. I will rebase this to the latest updates.
Regarding specifying the location for the folder. I thought about this and think it would be nice to have this set as an environment variable. I think the command line arguments can get a little messy. Especially depending how one is calling the pipeline, it might get long. What do you think? Then the user can set the environment variable however they want.

@dostuffthatmatters
Copy link
Member

dostuffthatmatters commented Oct 8, 2024

@cfleur great that it works now!

Agreed. Command line arguments might be messy because you also have to add it to every single CLI command/call. Environment variables is a good way of doing it.

@cfleur
Copy link
Contributor Author

cfleur commented Oct 16, 2024

Closing as PR #111 opened instead from different branch.

@cfleur cfleur closed this Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ability to specifiy alternate local config location
2 participants