Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

~4% perf regression in select-row benchmark due to on optimization #5009

Open
nolanlawson opened this issue Dec 5, 2024 · 0 comments
Open

Comments

@nolanlawson
Copy link
Collaborator

I did a Tachometer analysis of master vs the LWC version released for 252 (86682ec, v7.1.5).

Screenshot 2024-12-03 at 4 27 56 PM

Click to see

<style type="text/css"></style>

Benchmark Before (low) Before (high) Before (avg) After (low) After (high) After (avg) Delta (low) Delta (high) Delta (avg) Delta perc (low) Delta perc (high) Delta perc (avg)
engine-dom-append-rows-1k 17.07 17.22 17.14 16.82 16.93 16.88 -0.37 -0.17 -0.27 -0.02 -0.01 -2%
engine-dom-clear-rows-1k 4.81 4.92 4.86 4.69 4.79 4.74 -0.2 -0.05 -0.12 -0.04 -0.01 -3%
engine-dom-create-rows-10k 93.52 94.32 93.92 93.77 94.65 94.21 -0.31 0.89 0.29 0 0.01 0%
engine-dom-create-rows-1k 13.39 13.65 13.52 13.72 14 13.86 0.15 0.53 0.34 0.01 0.04 3%
engine-dom-expression 16.48 16.71 16.6 16.48 16.71 16.6 -0.16 0.16 0 -0.01 0.01 0%
engine-dom-light-slot-create-container-5k 261.04 262.23 261.64 263.85 265.1 264.47 1.97 3.7 2.84 0.01 0.01 1%
engine-dom-light-slot-update-slotted-content-5k 174.42 176.01 175.21 174.38 176.23 175.31 -1.12 1.31 0.1 -0.01 0.01 0%
engine-dom-light-styled-component-create-10k-same 91.52 91.94 91.73 92.54 92.92 92.73 0.72 1.29 1 0.01 0.01 1%
engine-dom-light-styled-component-create-1k-different 134.95 136.21 135.58 134.75 135.85 135.3 -1.11 0.56 -0.28 -0.01 0 0%
engine-dom-partial-update-1k 12.32 12.6 12.46 12.1 12.39 12.24 -0.42 -0.02 -0.22 -0.03 0 -2%
engine-dom-remove-row-1k 6.38 6.53 6.46 6.53 6.69 6.61 0.05 0.26 0.16 0.01 0.04 2%
engine-dom-replace-rows-1k 17.36 17.55 17.45 17.02 17.21 17.11 -0.47 -0.21 -0.34 -0.03 -0.01 -2%
engine-dom-select-row-1k 10.63 11.03 10.83 11.03 11.42 11.23 0.12 0.68 0.4 0.01 0.06 4%
engine-dom-ss-mutation-observer-10k 1348.58 1358.7 1353.64 1342.63 1352.78 1347.7 -13.1 1.23 -5.93 -0.01 0 0%
engine-dom-ss-slot-create-container-5k 358.68 360.76 359.72 360.66 362.57 361.62 0.49 3.3 1.9 0 0.01 1%
engine-dom-ss-slot-update-slotted-content-5k 217.65 218.87 218.26 220.02 221.27 220.65 1.51 3.26 2.39 0.01 0.01 1%
engine-dom-ss-styled-component-create-10k-same 147.12 148.04 147.58 147.9 148.84 148.37 0.13 1.45 0.79 0 0.01 1%
engine-dom-ss-styled-component-create-1k-different 153.97 155.42 154.7 153.61 154.68 154.15 -1.45 0.35 -0.55 -0.01 0 0%
engine-dom-ss-table-create-10k 139.92 140.99 140.45 139.37 140.32 139.84 -1.33 0.11 -0.61 -0.01 0 0%
engine-dom-styled-component-create-10k-same 164.93 166.31 165.62 167.57 169.08 168.33 1.68 3.73 2.71 0.01 0.02 2%
engine-dom-styled-component-create-1k-different 148.43 149.62 149.03 147.81 149.02 148.41 -1.47 0.23 -0.62 -0.01 0 0%
engine-dom-swap-rows-1k 10.54 10.83 10.68 10.71 10.98 10.84 -0.04 0.36 0.16 0 0.03 2%
engine-dom-table-create-10k 106.58 107.97 107.28 106.78 107.86 107.32 -0.84 0.93 0.04 -0.01 0.01 0%
engine-dom-tablecmp-append-1k 37.59 38.14 37.87 37.62 38.12 37.87 -0.37 0.37 0 -0.01 0.01 0%
engine-dom-tablecmp-create-10k 304.89 306.27 305.58 308.41 309.96 309.18 2.56 4.64 3.6 0.01 0.02 1%
engine-dom-tablecmp-create-1k 45.61 45.87 45.74 46.25 46.58 46.42 0.47 0.89 0.68 0.01 0.02 1%
engine-dom-trackedcmp-create-10k 271.56 272.94 272.25 274.46 275.79 275.13 1.92 3.84 2.88 0.01 0.01 1%
engine-dom-trackedcmp-update-10k 107.78 108.47 108.13 108.86 109.61 109.23 0.6 1.61 1.11 0.01 0.01 1%
engine-dom-tree-create-3k 101.53 102.06 101.8 102.95 103.48 103.22 1.05 1.8 1.42 0.01 0.02 1%
engine-dom-tree-remove-3k 47.45 47.87 47.66 47.42 47.93 47.68 -0.31 0.35 0.02 -0.01 0.01 0%
engine-dom-wireslist-create-10k 253.24 254.5 253.87 256.74 257.82 257.28 2.58 4.25 3.41 0.01 0.02 1%
engine-server-table-render-10k 56.16 56.38 56.27 55.47 55.66 55.57 -0.85 -0.56 -0.71 -0.02 -0.01 -1%
engine-server-tablecmp-render-10k 176.91 178.67 177.79 176.51 177.68 177.09 -1.76 0.36 -0.7 -0.01 0 0%
hydration-table-hydrate-1k 38.11 38.49 38.3 40.1 40.43 40.26 1.71 2.21 1.96 0.04 0.06 5%
hydration-tablecmp-hydrate-1k 39.08 39.39 39.23 41.8 42.13 41.96 2.5 2.96 2.73 0.06 0.08 7%

We've already fixed the last two (hydration) in #4996. For select-row, though, a git bisect revealed that #4468 was the source of the regression.

This is a bit surprising since this PR was supposed to be a perf improvement. Perhaps it improved perf in some benchmarks but regressed this one. We should do some analysis to see if overall this change is still worth it, or if there is some way to further optimize it.

git bisect script
#!/usr/bin/env bash

set -e
set -x

yarn
BENCHMARK_REF=86682ec8329eab989a6c9fd8a71eaf447e431eab BENCHMARK_SAMPLE_SIZE=400 BENCHMARK_TIMEOUT=60 yarn build:performance
cd packages/@lwc/perf-benchmarks

node ../../../node_modules/.bin/tach --config ./dist/__benchmarks__/engine-dom/js-framework-benchmark/select-row-1k.tachometer.json --json-file /tmp/bench.json

cd ../../..

highNum=$(cat /tmp/bench.json | jq .benchmarks[0].differences[1].percentChange.high | tr '.' ' ' | awk '{print $1}')
lowNum=$(cat /tmp/bench.json | jq .benchmarks[0].differences[1].percentChange.low | tr '.' ' ' | awk '{print $1}')

echo "Numbers are $lowNum to $highNum"

if (( $highNum >= 2 )); then
  if (( $lowNum >= 0 )); then
	echo "Fail"
    exit 1;
  fi
fi

echo "Success"

(I manually checked the git bisect run results, and yes, this script worked correctly.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant