Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix appending rnd query parameter to URLs in benchmark-server-timing #165

Merged
merged 1 commit into from
Nov 26, 2024

Conversation

westonruter
Copy link
Collaborator

@westonruter westonruter commented Nov 25, 2024

When running benchmark-server-timing on a URL I was seeing the expected results:

npm run research -- benchmark-server-timing -u 'http://localhost:10053/'

> research
> ./cli/run.mjs benchmark-server-timing -u http://localhost:10053/

╔════════════════════════════════════╤═════════════════════════╗
║ URL                                │ http://localhost:10053/ ║
╟────────────────────────────────────┼─────────────────────────╢
║ Success Rate                       │ 100%                    ║
╟────────────────────────────────────┼─────────────────────────╢
║ Response Time (median)             │ 45.71                   ║
╟────────────────────────────────────┼─────────────────────────╢
║ wp-before-template (median)        │ 19.61                   ║
╟────────────────────────────────────┼─────────────────────────╢
║ wp-template (median)               │ 22.75                   ║
╟────────────────────────────────────┼─────────────────────────╢
║ wp-optimization-detective (median) │ 3.41                    ║
╟────────────────────────────────────┼─────────────────────────╢
║ wp-total (median)                  │ 42.37                   ║
╚════════════════════════════════════╧═════════════════════════╝

However, when I tried adding a query parameter then I found the Success Rate to be 0%:

npm run research -- benchmark-server-timing -u 'http://localhost:10053/?optimization_detective_disabled=1'

> research
> ./cli/run.mjs benchmark-server-timing -u http://localhost:10053/?optimization_detective_disabled=1

╔════════════════════════╤═══════════════════════════════════════════════════════════╗
║ URL                    │ http://localhost:10053/?optimization_detective_disabled=1 ║
╟────────────────────────┼───────────────────────────────────────────────────────────╢
║ Success Rate           │ 0%                                                        ║
╟────────────────────────┼───────────────────────────────────────────────────────────╢
║ Response Time (median) │ 22.75                                                     ║
╟────────────────────────┼───────────────────────────────────────────────────────────╢
║ wp-total (median)      │ 19.66                                                     ║
╚════════════════════════╧═══════════════════════════════════════════════════════════╝

I thought this was a problem in Optimization Detective but no, it happens when there is any query param:

$ npm run research -- benchmark-server-timing -u 'http://localhost:10053/?foo=bar'

> research
> ./cli/run.mjs benchmark-server-timing -u http://localhost:10053/?foo=bar

╔════════════════════════╤═════════════════════════════════╗
║ URL                    │ http://localhost:10053/?foo=bar ║
╟────────────────────────┼─────────────────────────────────╢
║ Success Rate           │ 0%                              ║
╟────────────────────────┼─────────────────────────────────╢
║ Response Time (median) │ 30.43                           ║
╟────────────────────────┼─────────────────────────────────╢
║ wp-total (median)      │ 24.98                           ║
╚════════════════════════╧═════════════════════════════════╝

The problem is that the rnd query parameter is being added by appending ?rnd=... to the URL without accounting for whether there are existing query parameters in the URL. This fixes that problem, while also using Math.random() rather than an auto-incremented number. These changes had previously been done for benchmark-web-vitals in #60.

With the changes in this PR:

$ npm run research -- benchmark-server-timing -u 'http://localhost:10053/?optimization_detective_disabled=1'

> research
> ./cli/run.mjs benchmark-server-timing -u http://localhost:10053/?optimization_detective_disabled=1

╔═════════════════════════════╤═══════════════════════════════════════════════════════════╗
║ URL                         │ http://localhost:10053/?optimization_detective_disabled=1 ║
╟─────────────────────────────┼───────────────────────────────────────────────────────────╢
║ Success Rate                │ 100%                                                      ║
╟─────────────────────────────┼───────────────────────────────────────────────────────────╢
║ Response Time (median)      │ 48.76                                                     ║
╟─────────────────────────────┼───────────────────────────────────────────────────────────╢
║ wp-before-template (median) │ 25.4                                                      ║
╟─────────────────────────────┼───────────────────────────────────────────────────────────╢
║ wp-template (median)        │ 16.63                                                     ║
╟─────────────────────────────┼───────────────────────────────────────────────────────────╢
║ wp-total (median)           │ 42.03                                                     ║
╚═════════════════════════════╧═══════════════════════════════════════════════════════════╝

And:

$ npm run research -- benchmark-server-timing -u 'http://localhost:10053/' -n 100

> research
> ./cli/run.mjs benchmark-server-timing -u http://localhost:10053/ -n 100

╔════════════════════════════════════╤═════════════════════════╗
║ URL                                │ http://localhost:10053/ ║
╟────────────────────────────────────┼─────────────────────────╢
║ Success Rate                       │ 100%                    ║
╟────────────────────────────────────┼─────────────────────────╢
║ Response Time (median)             │ 32.13                   ║
╟────────────────────────────────────┼─────────────────────────╢
║ wp-before-template (median)        │ 12.12                   ║
╟────────────────────────────────────┼─────────────────────────╢
║ wp-template (median)               │ 18.32                   ║
╟────────────────────────────────────┼─────────────────────────╢
║ wp-optimization-detective (median) │ 3.17                    ║
╟────────────────────────────────────┼─────────────────────────╢
║ wp-total (median)                  │ 31.05                   ║
╚════════════════════════════════════╧═════════════════════════╝

And:

$ npm run research -- benchmark-server-timing --file=preload-bgimage-urls.txt --number 100

> research
> ./cli/run.mjs benchmark-server-timing --file=preload-bgimage-urls.txt --number 100

Benchmarking URL http://localhost:10053/?optimization_detective_disabled=1...Success.
Benchmarking URL http://localhost:10053/...Success.
╔════════════════════════════════════╤═══════════════════════════════════════════════════════════╤═════════════════════════╗
║ URL                                │ http://localhost:10053/?optimization_detective_disabled=1 │ http://localhost:10053/ ║
╟────────────────────────────────────┼───────────────────────────────────────────────────────────┼─────────────────────────╢
║ Success Rate                       │ 100%                                                      │ 100%                    ║
╟────────────────────────────────────┼───────────────────────────────────────────────────────────┼─────────────────────────╢
║ Response Time (median)             │ 29.25                                                     │ 33.57                   ║
╟────────────────────────────────────┼───────────────────────────────────────────────────────────┼─────────────────────────╢
║ wp-before-template (median)        │ 12.62                                                     │ 12.88                   ║
╟────────────────────────────────────┼───────────────────────────────────────────────────────────┼─────────────────────────╢
║ wp-template (median)               │ 15.52                                                     │ 19.09                   ║
╟────────────────────────────────────┼───────────────────────────────────────────────────────────┼─────────────────────────╢
║ wp-total (median)                  │ 28.02                                                     │ 32.38                   ║
╟────────────────────────────────────┼───────────────────────────────────────────────────────────┼─────────────────────────╢
║ wp-optimization-detective (median) │                                                           │ 3.24                    ║
╚════════════════════════════════════╧═══════════════════════════════════════════════════════════╧═════════════════════════╝

Copy link
Collaborator

@felixarntz felixarntz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@westonruter Great catch!

@felixarntz felixarntz merged commit 8333ae2 into main Nov 26, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants