feat(secret): secret management (part 1) add secret ref protos and referent count in meta catalog #17474
Conversation
| @@ -141,6 +144,7 @@ message SinkFormatDesc { | |||
| plan_common.EncodeType encode = 2; | |||
| map<string, string> options = 3; | |||
| optional plan_common.EncodeType key_encode = 4; | |||
| map<string, secret.SecretRef> secret_refs = 5; | |||
There was a problem hiding this comment.
Isn't it duplicate with the pre-defined one in sink proto?
There was a problem hiding this comment.
ditto, not sure about we need to introduce format-encode ones for Sink, cc @xiangjinwu
There was a problem hiding this comment.
Sink also uses schema.registry, so I guess it need secrets
If we know what StreamSourceInfo and SinkFormatDesc are for, while it's true not many have experience in schema registry, it's not really hard to understand the |
|
@kwannoel Is there any way we can bypass a fuzzy test? It's a big query causing stack overflow https://buildkite.com/risingwavelabs/pull-request/builds/52956#0190608e-9aca-43bb-95f4-1a2933141681 |
Add skipif madsim to your e2e test |
Hmm it's a fuzzy test generated by sqlsmith. |
In that case it indicates some sort of regression. Because it passes previously. You can remove the query from sqlsmith query snapshots here: https://github.com/risingwavelabs/sqlsmith-query-snapshots/blob/main/62/queries.sql. Just delete all the lines from https://github.com/risingwavelabs/sqlsmith-query-snapshots/blob/main/62/queries.sql#L40C1-L40C29 onwards. Including L40 itself. |
I see. I asked because I thought it's randomly generated at each run. I'll see what can cause this regression before remove it. Suprised this can happen consider I only added a few fields in plan node and catalog (and they are currently unused). cc @BugenZhao @st1page Got this |
…hao/add-secret-ref-proto
…hao/add-secret-ref-proto
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
This PR is part 1 of #17456
It will add secret ref protos and referent count in meta catalog and migration.
It only use a default dummy secret_ref in the proto for now.
The secret_ref should be in
Catalog
PbSource and PbSink have their secret_ref from with properties.
StreamSouceInfo and SinkDesc have their secret_ref from with properties of encode options (mostly used by schema registry).
Plan node
plan nodes should carry secret_refs from along with other properties and they don't need encode options.
Checklist
./risedev check(or alias,./risedev c)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.