Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1869802: Pivot converts 0 to None #2817

Open
AdamosX opened this issue Dec 30, 2024 · 2 comments
Open

SNOW-1869802: Pivot converts 0 to None #2817

AdamosX opened this issue Dec 30, 2024 · 2 comments
Assignees
Labels
bug Something isn't working local testing Local Testing issues/PRs status-triage_done Initial triage done, will be further handled by the driver team

Comments

@AdamosX
Copy link

AdamosX commented Dec 30, 2024

  1. What version of Python are you using?
    3.11.9

  2. What are the Snowpark Python and pandas versions in the environment?
    pandas==2.2.3
    snowflake-snowpark-python==1.26.0

  3. What did you do?

df = sess.create_dataframe([
    [1, "A", 10],
    [1, "B", 0],
    [2, "A", 11],
    [2, "B", 12]
])
pivoted = df.pivot(pivot_col="_2", values=["A", "B"]).function('min')("_3")
pivoted.collect()
  1. What did you expect to see?
    In Snowflake this results in: [Row(_1=1, 'A'=10, 'B'=0), Row(_1=2, 'A'=11, 'B'=12)]
    But in local testing: [Row(_1=1, A=10, B=None), Row(_1=2, A=11, B=12)]
    0 gets converted to None
@AdamosX AdamosX added bug Something isn't working local testing Local Testing issues/PRs needs triage Initial RCA is required labels Dec 30, 2024
@github-actions github-actions bot changed the title Pivot converts 0 to None SNOW-1869802: Pivot converts 0 to None Dec 30, 2024
@sfc-gh-sghosh sfc-gh-sghosh self-assigned this Jan 2, 2025
@sfc-gh-sghosh
Copy link

Hello @AdamosX ,

Thanks for raising the issue, we are able to reproduce it, we will work on eliminating it.

Regards,
Sujan

@sfc-gh-sghosh sfc-gh-sghosh added status-triage_done Initial triage done, will be further handled by the driver team and removed needs triage Initial RCA is required labels Jan 2, 2025
@AdamosX
Copy link
Author

AdamosX commented Jan 2, 2025

I did not mention that there are other differences:
In snowflake pivot adds '' to the column names in pivot and local testing does not.

The difference is evident when accessing the rows:

rows = pivoted.collect()
rows[0]["A"] # works with local testing session but not real session
rows[0]["'A'"] # works with real session but not local testing session

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working local testing Local Testing issues/PRs status-triage_done Initial triage done, will be further handled by the driver team
Projects
None yet
Development

No branches or pull requests

3 participants