Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qc fixes #401

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Qc fixes #401

wants to merge 10 commits into from

Conversation

leilabbb
Copy link
Contributor

update of the glider_qc.py module and functions

Improved or added Docstring to function, method, or class

Used the logging module to log message at the error level

Added  a report to functions' returns to log in the error

Used fill_value=np.int16(-999) for _FillValue for qartod variables. Using a flag value to _FillValue is not a good practice for quality control data.

Add a full array of 2 as a default for qartod variables. There are instances where array cannot be qc tested because of errors in attributes values or there are insufficient data or the array is empty.

Add a functionality to drop configuration specifications and skip QC if the specs cannot be calculated. This is for the function get_spike_thresholds and  get_rate_of_change_threshold

Changed the qartod method to calculate flags from XarrayStream to PandasStream to fix the array type compatability issues with the ioos-qc module.

Added the values mask check and invalis data check to the time array.
added the suspect_span to the density gross_range_test
benjwadams
benjwadams previously approved these changes Dec 2, 2024
@benjwadams benjwadams self-requested a review December 2, 2024 15:13
@benjwadams benjwadams dismissed their stale review December 2, 2024 15:13

Please address test failures

Copy link
Contributor

@benjwadams benjwadams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address test failures

minor fix to work with test_glider_qc.py
adjusted functions to match those of  the reworked glider_qc.py
fixed glider_qc/glider_qc.py:140: NameError: name 'ncfile' is not defined line
fixed tests/test_glider_qc.py:119: NameError: name 'normalize_variable' is not defined


fixed  tests/test_glider_qc.py:75: KeyError: 'gross_range_test'
Fixed tests/test_glider_qc.py:73: NameError: name 'tempvar' is not defined
Fixed tests/test_glider_qc.py:84: KeyError: 'gross_range_test'
@leilabbb leilabbb linked an issue Dec 19, 2024 that may be closed by this pull request
@leilabbb leilabbb self-assigned this Dec 19, 2024
@leilabbb
Copy link
Contributor Author

Address test failures

The unit tests were checked and modified to work with the new updates/changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment