Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Cassandra db schema on session initialization #5922

Merged

Conversation

akstron
Copy link
Contributor

@akstron akstron commented Sep 2, 2024

Create Schema (if not present) on Session Initialization

Once a session is established with cassandra db, the added code parses the template file containing queries for creating schema and create queries out of it. Post which it executes those queries to create the required types and tables.

Which problem is this PR solving?

Resolves #5797

Description of the changes

  • The PR includes the following changes:
    1. Embedding template files into binary
    1. Creation of database schema in initialization steps once session to database is established.

How was this change tested?

  • Schema rendering is being tested with unit test.
  • bash scripts/cassandra-integration-test.sh -s 4 v004 v2

Checklist

plugin/storage/cassandra/factory.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/factory.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/factory.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/factory.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/factory.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/factory.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/factory.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/factory.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/factory.go Outdated Show resolved Hide resolved
@akstron akstron force-pushed the create-database-scheme-cassandra branch 2 times, most recently from 69275fb to bcad4c0 Compare October 25, 2024 13:34
@akstron akstron marked this pull request as ready for review October 25, 2024 13:52
@akstron akstron requested a review from a team as a code owner October 25, 2024 13:52
@akstron akstron requested a review from albertteoh October 25, 2024 13:52
@akstron akstron requested a review from yurishkuro October 26, 2024 16:18
cmd/jaeger/config-cassandra.yaml Outdated Show resolved Hide resolved
pkg/cassandra/config/config.go Outdated Show resolved Hide resolved
pkg/cassandra/config/config.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/schema/schema.go Outdated Show resolved Hide resolved
@akstron akstron force-pushed the create-database-scheme-cassandra branch 2 times, most recently from 90368b1 to afc786d Compare October 28, 2024 10:50
pkg/cassandra/config/config.go Outdated Show resolved Hide resolved
pkg/cassandra/config/config.go Outdated Show resolved Hide resolved
pkg/cassandra/config/config.go Outdated Show resolved Hide resolved
plugin/storage/cassandra/schema/v004-go-tmpl-test.cql.tmpl Outdated Show resolved Hide resolved
plugin/storage/cassandra/schema/schema.go Outdated Show resolved Hide resolved
@akstron akstron changed the title Create Cassandra db schema on session initialization [WIP] Create Cassandra db schema on session initialization Nov 9, 2024
Copy link

codecov bot commented Nov 9, 2024

Codecov Report

Attention: Patch coverage is 64.15094% with 57 lines in your changes missing coverage. Please review.

Project coverage is 96.23%. Comparing base (2e5ed05) to head (b0ac38a).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
plugin/storage/cassandra/schema/schema.go 36.04% 55 Missing ⚠️
plugin/storage/cassandra/factory.go 94.11% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5922      +/-   ##
==========================================
- Coverage   96.43%   96.23%   -0.20%     
==========================================
  Files         355      356       +1     
  Lines       20165    20303     +138     
==========================================
+ Hits        19446    19539      +93     
- Misses        530      579      +49     
+ Partials      189      185       -4     
Flag Coverage Δ
badger_v1 8.11% <0.00%> (-0.11%) ⬇️
badger_v2 1.65% <0.00%> (-0.03%) ⬇️
cassandra-4.x-v1 ?
cassandra-4.x-v1-manual 14.28% <25.15%> (?)
cassandra-4.x-v2 ?
cassandra-4.x-v2-auto 1.59% <0.00%> (?)
cassandra-4.x-v2-manual 1.59% <0.00%> (?)
cassandra-5.x-v1 ?
cassandra-5.x-v1-manual 14.28% <25.15%> (?)
cassandra-5.x-v2 ?
cassandra-5.x-v2-auto 1.59% <0.00%> (?)
cassandra-5.x-v2-manual 1.59% <0.00%> (?)
elasticsearch-6.x-v1 18.14% <0.00%> (-0.26%) ⬇️
elasticsearch-7.x-v1 18.23% <0.00%> (-0.25%) ⬇️
elasticsearch-8.x-v1 18.38% <0.00%> (-0.25%) ⬇️
elasticsearch-8.x-v2 1.65% <0.00%> (-0.03%) ⬇️
grpc_v1 9.28% <0.00%> (-0.12%) ⬇️
grpc_v2 6.94% <0.00%> (-0.09%) ⬇️
kafka-v1 8.65% <0.00%> (-0.12%) ⬇️
kafka-v2 1.65% <0.00%> (-0.03%) ⬇️
memory_v2 1.65% <0.00%> (-0.03%) ⬇️
opensearch-1.x-v1 18.27% <0.00%> (-0.24%) ⬇️
opensearch-2.x-v1 18.27% <0.00%> (-0.25%) ⬇️
opensearch-2.x-v2 1.65% <0.00%> (-0.02%) ⬇️
tailsampling-processor 0.46% <0.00%> (-0.01%) ⬇️
unittests 95.17% <63.52%> (-0.18%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@akstron akstron changed the title [WIP] Create Cassandra db schema on session initialization Create Cassandra db schema on session initialization Nov 9, 2024
@akstron akstron changed the title Create Cassandra db schema on session initialization [WIP] Create Cassandra db schema on session initialization Nov 10, 2024
@akstron akstron marked this pull request as draft November 10, 2024 05:21
…ution for initialize database

Signed-off-by: Alok Kumar Singh <[email protected]>
@yurishkuro
Copy link
Member

51.06% of diff hit (target 95.00%)

the code coverage for new code is quite low, can you add tests for the error edge cases?

Signed-off-by: Alok Kumar Singh <[email protected]>
change default for v2 to auto-create schema

Signed-off-by: Yuri Shkuro <[email protected]>
yurishkuro and others added 2 commits November 30, 2024 12:58
Signed-off-by: Yuri Shkuro <[email protected]>
Signed-off-by: Yuri Shkuro <[email protected]>
@yurishkuro
Copy link
Member

@akstron I refactored the changes a bit to move session and schema creation into the actual storage implementation and leave the config package to only deal with configuration, not with creating live sessions.

Signed-off-by: Yuri Shkuro <[email protected]>
Signed-off-by: Yuri Shkuro <[email protected]>
Signed-off-by: Yuri Shkuro <[email protected]>
Signed-off-by: Yuri Shkuro <[email protected]>
yurishkuro and others added 2 commits November 30, 2024 14:59
Signed-off-by: Yuri Shkuro <[email protected]>
Signed-off-by: Yuri Shkuro <[email protected]>
@yurishkuro yurishkuro merged commit e9fac05 into jaegertracing:main Nov 30, 2024
53 of 54 checks passed
@yurishkuro
Copy link
Member

🎉 🎉 🎉 great job @akstron

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/storage changelog:new-feature Change that should be called out as new feature in CHANGELOG storage/cassandra
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create database schema in Cassandra automatically
3 participants