-
Notifications
You must be signed in to change notification settings - Fork 595
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sink emitting more rows than expected, followup of #10853 #13025
Comments
I guess the key is expected to be pim_catalog_product.id instead of the full raw_values. Would you please provide the complete explain CREATE SINK sink_name .... Also the table's definition in the downstream PG. |
for sure, sorry for not including it (I've got the source code here just in case: https://github.com/docteurklein/risingwave-demo/blob/main/rw/product.sql) sink DDLcreate sink product_value_edited from pim1.product_value
with (
connector='kafka',
type='upsert',
primary_key='product_id, attribute, channel, locale',
-- force_append_only='true',
properties.bootstrap.server='redpanda:9092',
topic='product_value_edited'
); explain create sink
mysql source DDLcreate table pim_catalog_product (
uuid binary(16),
family_id int,
product_model_id int,
family_variant_id int,
id int,
is_enabled bool,
identifier text,
raw_values json,
created timestamp,
updated timestamp,
primary key (uuid)
);
insert into pim_catalog_product (uuid, id, family_id, raw_values) values
(UUID_TO_BIN(uuid()), 1, 1, '{}'); I'll gladly provide anything that could help, Thanks for the quick answer! |
I guess you should change the primary key to |
Looks like that the Then, during We may support update compaction by sink pk when sinking in the future (still investigating), but currently it seems to be expected behavior (as long as the final output is correct). If the final result is actually wrong, please notify us, may need further debugging. |
Sorry, no computer access right now, but yes that's exactly what @stdrc described. The output always contains correct values (although some are extraneous since some rows of the mat view didn't change (like name and desc2 in the example). Ok so I'll wait for update compaction then. Thanks for the clarifications! |
I think it has been fixed after #15345 and ok in release v1.8 |
Closing this issue since the fix has been merged. |
Describe the bug
followup of #10853
looks like I'm still getting extraneous sink outputs when changing only 1 element of a json structure containing siblings.
version: PostgreSQL 9.5-RisingWave-1.3.0-alpha (bb2319e)
on mysql source
expected output:
actual output
The text was updated successfully, but these errors were encountered: