Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: bump sqllogictest & fix backwards-compat-test/e2e-tests #14354

Merged
merged 12 commits into from
Jan 5, 2024

Conversation

xxchan
Copy link
Member

@xxchan xxchan commented Jan 4, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

  • feat: report error when include empty result risinglightdb/sqllogictest-rs#203 to help found issues. Previously empty includes are silently ignored.
  • For backwards-compat-test, the delete.slt is not included at all.
    • But previously tests passed. I think it's because the writes are not flushed. (default barrier_interval_ms is 1000)
    • If we only add flush after insert, but not after delete, nexmark/tpch q0,1,2 passed, but q3 failed. This is a little strange. Is it because there are some more deletes in the second epoch?

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

@xxchan xxchan added the ci/run-backwards-compat-tests Run backwards compatibility tests in your PR. label Jan 4, 2024
@github-actions github-actions bot added the component/test Test related issue. label Jan 4, 2024
@xxchan
Copy link
Member Author

xxchan commented Jan 4, 2024

Indeed. (Why this passed before?)

image

@xxchan xxchan changed the title test: IGNORE test: fix backwards-compat-test & bump sqllogictest Jan 4, 2024
@xxchan
Copy link
Member Author

xxchan commented Jan 4, 2024

This is very strange. Q3 failed

image

@xxchan xxchan changed the title test: fix backwards-compat-test & bump sqllogictest test: bump sqllogictest & fix backwards-compat-test/e2e-tests Jan 4, 2024
@xxchan
Copy link
Member Author

xxchan commented Jan 4, 2024

Tests passed now... Let's think what happened.

@xxchan xxchan requested review from BugenZhao and kwannoel January 4, 2024 08:26
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stdrc I just ran gen.py. Is the new file expected?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that's because you renamed row_number_old.part to row_number_old.slt.part. gen.py only generates for .slt and .slt.part files.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it should be renamed. It's leaved here just for reference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original error before my change is this. Can you help fix that?
image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the way you like

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK sure

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, interesting...Seems I did indeed want it to be included...

Comment on lines +5 to +6
statement ok
flush;
Copy link
Member Author

@xxchan xxchan Jan 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I don't add this flush, q0,1,2 succeed, but q3 failed. Why? 🤡

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's ordering of DELETE and INSERT, I don't understand why q0,1,2 succeed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q0,1,2 seem only depend on bid while q3 also depends on auction

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what does this imply? 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IDK, just seems to be explainable if bid table is OK but auction table is not ready

Copy link
Member Author

@xxchan xxchan Jan 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can reproduce this locally. It fails sometimes, and person is empty when it fails. bid and auction is fine. 🤡

include ../e2e_test/nexmark/create_tables.slt.part

include ../e2e_test/nexmark/insert_person.slt.part
include ../e2e_test/nexmark/insert_auction.slt.part
include ../e2e_test/nexmark/insert_bid.slt.part

include ../e2e_test/streaming/nexmark/views/q3.slt.part
include ../e2e_test/streaming/nexmark/q3.slt.part

statement ok
DELETE FROM person;

statement ok
DELETE FROM auction;

statement ok
DELETE FROM bid;

include ../e2e_test/nexmark/insert_person.slt.part
include ../e2e_test/nexmark/insert_auction.slt.part
include ../e2e_test/nexmark/insert_bid.slt.part

statement ok
flush;

include ../e2e_test/streaming/nexmark/q3.slt.part

statement ok
drop materialized view nexmark_q3;

include ../e2e_test/nexmark/drop_tables.slt.part

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had thought that when DML query returns, it will take effect, although not immediately visible. But it seems not correct? 🤔

Copy link
Member Author

@xxchan xxchan Jan 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After the DML returns, the data is only sent to some internal channels, not handled by the table yet, and there are multiple channels so the order is not known.

Copy link
Member

@stdrc stdrc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -21,7 +21,7 @@ SET RW_IMPLICIT_FLUSH TO true;
include ../batch/types/boolean.slt.part
include ../batch/types/cast.slt.part
include ../batch/types/date.slt
include ../batch/types/intercal.slt.part
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄

Comment on lines +5 to +6
statement ok
flush;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q0,1,2 seem only depend on bid while q3 also depends on auction

Copy link
Contributor

@kwannoel kwannoel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the fix.

@xxchan xxchan added this pull request to the merge queue Jan 5, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 5, 2024
@xxchan xxchan added this pull request to the merge queue Jan 5, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 5, 2024
@xxchan xxchan added this pull request to the merge queue Jan 5, 2024
Merged via the queue into main with commit cb825f0 Jan 5, 2024
38 of 39 checks passed
@xxchan xxchan deleted the xxchan/vitreous-crane branch January 5, 2024 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/run-backwards-compat-tests Run backwards compatibility tests in your PR. component/test Test related issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants