Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: auto generate virtual columns for Variant data type #17083

Open
3 tasks
b41sh opened this issue Dec 19, 2024 · 0 comments
Open
3 tasks

Feature: auto generate virtual columns for Variant data type #17083

b41sh opened this issue Dec 19, 2024 · 0 comments
Assignees
Labels
C-feature Category: feature

Comments

@b41sh
Copy link
Member

b41sh commented Dec 19, 2024

Summary

Virtual columns can greatly improve performance by extracting internal json fields and storing them separately, but now virtual columns need to be manually created by users, which is inconvenient to use. We need to support automatic generation. By extracting the schema of each json fields in the block, we can determine whether it is the same type and automatically generate virtual columns.

  • Add the address of the virtual column file at BlockMeta to determine whether the file is generated during refresh.
  • Derive the schema of the json fields to determine whether each field has the same type, and if so, it is suitable to generate virtual columns.
  • Set the generation rule to automatically refresh to generate virtual columns.
@b41sh b41sh added the C-feature Category: feature label Dec 19, 2024
@b41sh b41sh self-assigned this Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-feature Category: feature
Projects
None yet
Development

No branches or pull requests

1 participant