background DDL test in madsim should consider whether the statement is skipped.

[kwannoel]

Describe the bug

Seems like it was intended for this test to be skipped in madsim, but it's not.

Should it be skipped 🤔

Error message/log

2022-10-11T07:23:29.430473Z DEBUG risingwave_simulation::slt: Record Statement { loc: Location { file: "e2e_test/streaming/bug_fixes/stack_overflow_17342.slt", line: 10, upper: None }, conditions: [SkipIf { label: "madsim" }], connection: Default, sql: "CREATE MATERIALIZED VIEW mv AS\nSELECT\n    count(*)\nFROM\n    t\nFULL OUTER JOIN t t1 USING (v)\nFULL OUTER JOIN t t2 USING (v)\nFULL OUTER JOIN t t3 USING (v)\nFULL OUTER JOIN t t4 USING (v)\nFULL OUTER JOIN t t5 USING (v)\nFULL OUTER JOIN t t6 USING (v)\nFULL OUTER JOIN t t7 USING (v)\nFULL OUTER JOIN t t8 USING (v)\nFULL OUTER JOIN t t9 USING (v)\nFULL OUTER JOIN t t10 USING (v)\nFULL OUTER JOIN t t11 USING (v)\nFULL OUTER JOIN t t12 USING (v)\nFULL OUTER JOIN t t13 USING (v)\nFULL OUTER JOIN t t14 USING (v)\nFULL OUTER JOIN t t15 USING (v)\nFULL OUTER JOIN t t16 USING (v)\nFULL OUTER JOIN t t17 USING (v)\nFULL OUTER JOIN t t18 USING (v)\nFULL OUTER JOIN t t19 USING (v)\nFULL OUTER JOIN t t20 USING (v)\nFULL OUTER JOIN t t21 USING (v)\nFULL OUTER JOIN t t22 USING (v)\nFULL OUTER JOIN t t23 USING (v)\nFULL OUTER JOIN t t24 USING (v)\nFULL OUTER JOIN t t25 USING (v)\nFULL OUTER JOIN t t26 USING (v)\nFULL OUTER JOIN t t27 USING (v)\nFULL OUTER JOIN t t28 USING (v)\nFULL OUTER JOIN t t29 USING (v)\nFULL OUTER JOIN t t30 USING (v)\nFULL OUTER JOIN t t31 USING (v)\nFULL OUTER JOIN t t32 USING (v)\nFULL OUTER JOIN t t33 USING (v)\nFULL OUTER JOIN t t34 USING (v)\nFULL OUTER JOIN t t35 USING (v)\nFULL OUTER JOIN t t36 USING (v)\nFULL OUTER JOIN t t37 USING (v)\nFULL OUTER JOIN t t38 USING (v)\nFULL OUTER JOIN t t39 USING (v)\nFULL OUTER JOIN t t40 USING (v)\nFULL OUTER JOIN t t41 USING (v)\nFULL OUTER JOIN t t42 USING (v)\nFULL OUTER JOIN t t43 USING (v)\nFULL OUTER JOIN t t44 USING (v)\nFULL OUTER JOIN t t45 USING (v)\nFULL OUTER JOIN t t46 USING (v)\nFULL OUTER JOIN t t47 USING (v)\nFULL OUTER JOIN t t48 USING (v)\nFULL OUTER JOIN t t49 USING (v)\nFULL OUTER JOIN t t50 USING (v)\nFULL OUTER JOIN t t51 USING (v)\nFULL OUTER JOIN t t52 USING (v)\nFULL OUTER JOIN t t53 USING (v)\nFULL OUTER JOIN t t54 USING (v)\nFULL OUTER JOIN t t55 USING (v)\nFULL OUTER JOIN t t56 USING (v)\nFULL OUTER JOIN t t57 USING (v)\nFULL OUTER JOIN t t58 USING (v)\nFULL OUTER JOIN t t59 USING (v)\nFULL OUTER JOIN t t60 USING (v)\nFULL OUTER JOIN t t61 USING (v)\nFULL OUTER JOIN t t62 USING (v)\nFULL OUTER JOIN t t63 USING (v)\nFULL OUTER JOIN t t64 USING (v)\nFULL OUTER JOIN t t65 USING (v)\nFULL OUTER JOIN t t66 USING (v)\nFULL OUTER JOIN t t67 USING (v)\nFULL OUTER JOIN t t68 USING (v)\nFULL OUTER JOIN t t69 USING (v)\nFULL OUTER JOIN t t70 USING (v)\nFULL OUTER JOIN t t71 USING (v)\nFULL OUTER JOIN t t72 USING (v)\nFULL OUTER JOIN t t73 USING (v)\nFULL OUTER JOIN t t74 USING (v)\nFULL OUTER JOIN t t75 USING (v)\nFULL OUTER JOIN t t76 USING (v)\nFULL OUTER JOIN t t77 USING (v)\nFULL OUTER JOIN t t78 USING (v)\nFULL OUTER JOIN t t79 USING (v)\nFULL OUTER JOIN t t80 USING (v)\nFULL OUTER JOIN t t81 USING (v)\nFULL OUTER JOIN t t82 USING (v)\nFULL OUTER JOIN t t83 USING (v)\nFULL OUTER JOIN t t84 USING (v)\nFULL OUTER JOIN t t85 USING (v)\nFULL OUTER JOIN t t86 USING (v)\nFULL OUTER JOIN t t87 USING (v)\nFULL OUTER JOIN t t88 USING (v)\nFULL OUTER JOIN t t89 USING (v)\nFULL OUTER JOIN t t90 USING (v)\nFULL OUTER JOIN t t91 USING (v)\nFULL OUTER JOIN t t92 USING (v)\nFULL OUTER JOIN t t93 USING (v)\nFULL OUTER JOIN t t94 USING (v)\nFULL OUTER JOIN t t95 USING (v)\nFULL OUTER JOIN t t96 USING (v)\nFULL OUTER JOIN t t97 USING (v)\nFULL OUTER JOIN t t98 USING (v)\n;", expected: Ok } finished in 0ns
2022-10-11T07:23:29.430473Z DEBUG risingwave_simulation::slt: Retry for background ddl iteration=2
2022-10-11T07:23:29.507369Z ERROR risingwave_simulation::slt: failed to wait for background mv to finish creating iteration=2 err=expected 1 row in pg_matviews, got Ok(
    0,
) instead for mv

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response

[BugenZhao]

It did get skipped, but we were not aware of that and still tried to wait for the MV to be created.

risingwave/src/tests/simulation/src/slt.rs

Lines 340 to 386 in 209fd32

	match tester
	.run_async(record.clone())
	.timed(|_res, elapsed| {
	tracing::debug!("Record {:?} finished in {:?}", record, elapsed)
	})
	.await
	{
	Ok(_) => {
	// For background ddl
	if let SqlCmd::CreateMaterializedView { ref name } = cmd
	&& background_ddl_enabled
	&& !matches!(
	record,
	Record::Statement {
	expected: StatementExpect::Error(_),
	..
	} | Record::Query {
	expected: QueryExpect::Error(_),
	..
	}
	)
	{
	tracing::debug!(iteration = i, "Retry for background ddl");
	match wait_background_mv_finished(name).await {
	Ok(_) => {
	tracing::debug!(
	iteration = i,
	"Record with background_ddl {:?} finished",
	record
	);
	break;
	}
	Err(err) => {
	tracing::error!(
	iteration = i,
	?err,
	"failed to wait for background mv to finish creating"
	);
	if i >= MAX_RETRY {
	panic!("failed to run test after retry {i} times, error={err:#?}");
	}
	continue;
	}
	}
	}
	break;
	}

risinglightdb/sqllogictest-rs#219 could help to retrieve such information.

[BugenZhao]

Workarounded in #17440