Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] Test data caching and unzipping #83

Merged
merged 1 commit into from
Dec 18, 2024
Merged

Conversation

AlexVCaron
Copy link
Collaborator

@AlexVCaron AlexVCaron commented Dec 18, 2024

There is problems currently when test data starts to download but then an error occurs in the code or the workflow. It was hard to implement in the current form of the load_test_data workflow. I revamped the structure that handled the cache, using CRUD operations (create, read, update, delete), so every corner cases are correctly handled by implementation. There are try/catch everywhere to make sure that the cache doesn't get polluted, so we should be ok from now on.

For reference, it's kind of what I implemented in Scilpy for fetch_data, but better 😛

Also, in addition :

  • Added pdiff and configured the devcontainer to print the differences in nf-test results better

Bug category

  • Critical (some functionalities is not working at all)
  • Major (something is not working as expected)
  • Minor (something but could be improved)
  • Trivial (documentation needs correcting and other non-functional issues)

Describe the bug

Bad data gets stuck in cache.

Steps to reproduce the bug

  1. Delete the cache (in ~/.local/share/nf-test-archives)
  2. Start a test and kill it fast. Then again, a few time. You'll corrupt the cache eventually doing this.
  3. Start a test and let it run. It should error out. If not, sart again at 2.

@AlexVCaron
Copy link
Collaborator Author

@gagnonanthony this should resolve the problems with preproc_t1 leading to errors. I'll update you on that once it works on #82

@arnaudbore arnaudbore added this pull request to the merge queue Dec 18, 2024
Merged via the queue into scilus:main with commit 8702891 Dec 18, 2024
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants