Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Skip metrics on release branches #4312

Merged
merged 5 commits into from
Nov 26, 2024

Conversation

krystofwoldrich
Copy link
Member

@krystofwoldrich krystofwoldrich commented Nov 26, 2024

This PR adds a skip for Metrics perf jobs on release branches, these jobs are flaky (negative app start diff) and thus can block releases for "no reason".

#skip-changelog

Example of tests run with test condition

Copy link
Contributor

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 446.38 ms 434.64 ms -11.74 ms
Size 17.74 MiB 20.09 MiB 2.35 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
8ae23a7 526.83 ms 513.38 ms -13.45 ms
4297324 536.61 ms 542.48 ms 5.87 ms
dadc233+dirty 333.78 ms 343.94 ms 10.16 ms
c2a4e9b 480.42 ms 485.60 ms 5.18 ms
9433f35 347.64 ms 356.22 ms 8.58 ms
9a3ca65+dirty 326.93 ms 330.14 ms 3.21 ms
7bc4d75 488.76 ms 473.28 ms -15.48 ms
0d3e677 422.82 ms 411.90 ms -10.92 ms
80b2ce3 385.02 ms 387.36 ms 2.34 ms
1d86dd6 405.14 ms 411.06 ms 5.92 ms

App size

Revision Plain With Sentry Diff
8ae23a7 17.74 MiB 20.07 MiB 2.34 MiB
4297324 17.74 MiB 20.08 MiB 2.34 MiB
dadc233+dirty 17.73 MiB 19.75 MiB 2.02 MiB
c2a4e9b 17.73 MiB 20.06 MiB 2.33 MiB
9433f35 17.73 MiB 19.81 MiB 2.08 MiB
9a3ca65+dirty 17.73 MiB 20.04 MiB 2.31 MiB
7bc4d75 17.74 MiB 20.08 MiB 2.34 MiB
0d3e677 17.74 MiB 20.07 MiB 2.34 MiB
80b2ce3 17.73 MiB 19.75 MiB 2.02 MiB
1d86dd6 17.73 MiB 19.86 MiB 2.12 MiB

Copy link
Collaborator

@antonis antonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change LGTM and metrics are not skipped on non-release branches(e.g. this).

@krystofwoldrich krystofwoldrich merged commit d56d245 into main Nov 26, 2024
55 of 57 checks passed
@krystofwoldrich krystofwoldrich deleted the kw/ci/skip-metric-on-release branch November 26, 2024 10:20
Copy link
Contributor

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1220.56 ms 1221.39 ms 0.83 ms
Size 2.36 MiB 3.11 MiB 759.86 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
e2b64fe+dirty 1232.22 ms 1255.20 ms 22.98 ms
4cc5c27+dirty 1211.45 ms 1214.60 ms 3.16 ms
1c65324+dirty 1235.17 ms 1235.08 ms -0.09 ms
cdf2f33+dirty 1227.71 ms 1233.94 ms 6.22 ms
e1ea4a8+dirty 1233.49 ms 1237.71 ms 4.22 ms
dadc233+dirty 1223.20 ms 1236.88 ms 13.68 ms
ed3d77e+dirty 1214.61 ms 1205.83 ms -8.78 ms
2ec71da+dirty 1225.85 ms 1231.57 ms 5.72 ms
9cd0e9f+dirty 1224.94 ms 1239.88 ms 14.94 ms
0db0c72+dirty 1275.02 ms 1285.84 ms 10.82 ms

App size

Revision Plain With Sentry Diff
e2b64fe+dirty 2.36 MiB 2.85 MiB 495.80 KiB
4cc5c27+dirty 2.36 MiB 3.04 MiB 698.52 KiB
1c65324+dirty 2.36 MiB 3.04 MiB 698.64 KiB
cdf2f33+dirty 2.36 MiB 3.10 MiB 751.38 KiB
e1ea4a8+dirty 2.36 MiB 3.10 MiB 752.31 KiB
dadc233+dirty 2.36 MiB 2.84 MiB 486.85 KiB
ed3d77e+dirty 2.36 MiB 3.10 MiB 753.32 KiB
2ec71da+dirty 2.36 MiB 3.13 MiB 784.66 KiB
9cd0e9f+dirty 2.36 MiB 3.08 MiB 735.56 KiB
0db0c72+dirty 2.36 MiB 2.84 MiB 487.01 KiB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants