Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: fix X_{suffix} validation and add test coverage #635

Merged
merged 5 commits into from
Sep 20, 2023

Conversation

joyceyan
Copy link
Contributor

@joyceyan joyceyan commented Sep 19, 2023

Reason for Change

#590

Changes

  • fixes some of the issues caught by @jahilton:
    • only fail if all values are NaN, not if any values are NaN
    • having NaN should not result in the infinity error message getting picked up
    • validate that suffixes are at least one character
    • validate that keys don't have whitespace
    • gracefully fail if values have strings, aka don't fail on the isinf call
  • adds test coverage for all of the new validation things added

Testing

  • Either list QA steps or reasoning you feel QA is unnecessary
  • Reminder For CLI changes: upon merge, contact Lattice for final sign-off. Do not release a new cellxgene-schema
    version to PyPI without explicit QA + sign-off from Lattice on all functional CLI changes. They may install the package
    version at HEAD of main with
pip install git+https://github.com/chanzuckerberg/single-cell-curation/@main#subdirectory=cellxgene_schema_cli

Notes for Reviewer

@codecov
Copy link

codecov bot commented Sep 19, 2023

Codecov Report

Merging #635 (b3ab1c8) into main (1f482ae) will increase coverage by 0.21%.
Report is 3 commits behind head on main.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #635      +/-   ##
==========================================
+ Coverage   83.23%   83.44%   +0.21%     
==========================================
  Files          19       19              
  Lines        1724     1728       +4     
==========================================
+ Hits         1435     1442       +7     
+ Misses        289      286       -3     
Flag Coverage Δ
unittests 83.44% <100.00%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
cellxgene_schema_cli/cellxgene_schema/validate.py 94.19% <100.00%> (+0.59%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@@ -822,6 +822,9 @@ def _validate_embedding_dict(self):

obsm_with_x_prefix = 0
for key, value in self.adata.obsm.items():
if " " in key:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think any keys that have whitespace in them should be invalid, regardless of whether it's an X_{suffix} key

self.validator.errors,
["ERROR: Embedding key X_ umap has whitespace in it, please remove it."],
)

def test_obsm_shape(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a test for the other condition on this if conditional? i.e. must have same number of rows as X

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i updated for the case where the number of rows is not the same as the number of cells. it actually seems to fail when setting the key, not when running the validation. but the test should reflect that

Copy link
Contributor

@nayib-jose-gloria nayib-jose-gloria left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, thank you! flagged 1 test scenario I think we're missing (from before these additions, but would be good to get coverage on while we're changing this block)

@joyceyan joyceyan merged commit 65e8c59 into main Sep 20, 2023
8 checks passed
@joyceyan joyceyan deleted the joyce/test-suffix branch September 20, 2023 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants