k6를 사용한 서버 부하 테스트
고려한 시나리오:
- 키워드 검색 (800명)
- 전체 플로우 (로그인, 리스트 조회, 상세조회, 삭제)
800명을 기준으로 진행했을 때 평균 13.35s 의 응답시간 이 조건으로 테스트가 불가능할것으로 예상 성능 개선을 염두에 두고 테스트 보류
- VUs = 50
- 스레드도 전체 200개를 전부 사용 X
- hikariCP connectionPool 도 10개 전부 사용 X
- 응답 시간 P(95): 43.06ms
- RPS: 65
값 조정을 하지 않아도 충분한 성능이라 판단.
부하가 발생하는 상황을 판단하기 위해 500명으로 늘려봄
-
VUs = 500
- 7번의 request timeout 발생
- 평균 2.07s, 중위값 1.39s 의 응답시간
- 메트릭의 TPS 가 수렴하지 않음
- 부하 테스트를 진행하기에 너무 많은 수라 판단
-
VUs = 200
- 3번의 request timeout 발생
- 응답시간: avg=416.53ms, med=214.16ms, p(95)=1.21s
- RPS 115
- 이 수치를 기준으로 성능 개선 시작
maximum-pool-size: 5
minimum-idle: 2
data-source-properties:
cachePrepStmts: true
prepStmtCacheSize: 250
prepStmtCacheSqlLimit: 2048
useServerPrepStmts: true
성능이 개선되지 않았음
- 📉 RPS: 115 -> 107
- 📈 duration P(95): 1.21s -> 1.75s
- 📈 connection Acquire Time(max): 11.1ms -> 24.5ms
- 📉 connection Usage Time(avg): 15.9ms -> 10ms
- 서버의 TPS 수렴하지 않음
성능이 유의미하게 개선
- 📈 TPS: 115 -> 160
- 📈 duration P(95): 1.21s -> 1.29S
- 📉 connection Acquire Time(max): 11.1ms -> 8.44ms
- 📉 connection Usage Time(avg): 15.9ms -> 30.1ms
- 메트릭 TPS 수렴\
server:
tomcat:
threads:
max: 100
data-source-properties:
cachePrepStmts: true
prepStmtCacheSize: 250
prepStmtCacheSqlLimit: 2048
useServerPrepStmts: true
성능이 유의미하게 개선
- 📈 TPS: 115 -> 157
- 📈 duration P(95): 1.21s -> 1.32
- 📈 connection Acquire Time(max): 11.1ms -> 17.4ms
- 📈 connection Usage Time(avg): 15.9ms -> 39.8ms
- 다만, DB 설정은 default 가 더 좋음
- 분산 환경이기에 VUs 200 으로 부하 테스트가 적절하지 못함.
- VUs 400 으로 증가시켜 부하테스트 진행
k6 run group_test.js
/\ Grafana /‾‾/
/\ / \ |\ __ / /
/ \/ \ | |/ / / ‾‾\
/ \ | ( | (‾) |
/ __________ \ |_|\_\ \_____/
execution: local
script: group_test.js
output: -
scenarios: (100.00%) 1 scenario, 400 max VUs, 16m30s max duration (incl. graceful stop):
* default: Up to 400 looping VUs for 16m0s over 2 stages (gracefulRampDown: 30s, gracefulStop: 30s)
WARN[0151] The test has generated metrics with 100102 unique time series, which is higher than the suggested limit of 100000 and could cause high memory usage. Consider not using high-cardinality values like unique IDs as metric tags or, if you need them in the URL, use the name metric tag or URL grouping. See https://grafana.com/docs/k6/latest/using-k6/tags-and-groups/ for details. component=metrics-engine-ingester
WARN[0268] The test has generated metrics with 200065 unique time series, which is higher than the suggested limit of 100000 and could cause high memory usage. Consider not using high-cardinality values like unique IDs as metric tags or, if you need them in the URL, use the name metric tag or URL grouping. See https://grafana.com/docs/k6/latest/using-k6/tags-and-groups/ for details. component=metrics-engine-ingester
WARN[0503] The test has generated metrics with 400018 unique time series, which is higher than the suggested limit of 100000 and could cause high memory usage. Consider not using high-cardinality values like unique IDs as metric tags or, if you need them in the URL, use the name metric tag or URL grouping. See https://grafana.com/docs/k6/latest/using-k6/tags-and-groups/ for details. component=metrics-engine-ingester
█ setup
█ login
✓ login successful
█ search
✓ request successful
█ create
✓ request successful
█ delete
✓ request successful
checks.........................: 100.00% 527479 out of 527479
create_template................: avg=41.739703 min=15.003 med=34.955 max=5257.001 p(90)=60.4808 p(95)=77.742
data_received..................: 456 MB 473 kB/s
data_sent......................: 52 MB 54 kB/s
delete_template................: avg=35.565071 min=14.355 med=30.042 max=5199.127 p(90)=50.66 p(95)=63.1708
get_detail_template............: avg=33.948449 min=11.269 med=27.897 max=7145.102 p(90)=41.5848 p(95)=51.7198
get_list.......................: avg=46.386152 min=11.843 med=31.303 max=7193.304 p(90)=56.225 p(95)=77.7818
get_list_by_category...........: avg=44.017326 min=11.613 med=31.027 max=7659.302 p(90)=54.1706 p(95)=71.6366
get_list_by_tag................: avg=36.196964 min=11.109 med=26.756 max=7140.658 p(90)=44.8768 p(95)=58.226
group_duration.................: avg=79.85ms min=14.58ms med=42.08ms max=9.39s p(90)=140.21ms p(95)=169.83ms
http_req_blocked...............: avg=19.41µs min=0s med=0s max=162.73ms p(90)=1µs p(95)=1µs
http_req_connecting............: avg=5.97µs min=0s med=0s max=63.79ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=39.64ms min=11.1ms med=30.08ms max=7.65s p(90)=52.05ms p(95)=67.79ms
{ expected_response:true }...: avg=39.64ms min=11.1ms med=30.08ms max=7.65s p(90)=52.05ms p(95)=67.79ms
http_req_failed................: 0.00% 0 out of 527479
http_req_receiving.............: avg=218.75µs min=4µs med=50µs max=103.95ms p(90)=555µs p(95)=859µs
http_req_sending...............: avg=110.02µs min=9µs med=59µs max=33.59ms p(90)=168µs p(95)=289µs
http_req_tls_handshaking.......: avg=12.11µs min=0s med=0s max=116.38ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=39.31ms min=63µs med=29.75ms max=7.65s p(90)=51.75ms p(95)=67.47ms
http_reqs......................: 527479 546.95233/s
iteration_duration.............: avg=4.24s min=4.09s med=4.18s max=13.48s p(90)=4.29s p(95)=4.37s
iterations.....................: 87913 91.158549/s
vus............................: 39 min=6 max=400
vus_max........................: 400 min=400 max=400
running (16m04.4s), 000/400 VUs, 87913 complete and 0 interrupted iterations
default ✓ [======================================] 000/400 VUs 16m0s
k6 run group_test.js
/\ Grafana /‾‾/
/\ / \ |\ __ / /
/ \/ \ | |/ / / ‾‾\
/ \ | ( | (‾) |
/ __________ \ |_|\_\ \_____/
execution: local
script: group_test.js
output: -
scenarios: (100.00%) 1 scenario, 400 max VUs, 16m30s max duration (incl. graceful stop):
* default: Up to 400 looping VUs for 16m0s over 2 stages (gracefulRampDown: 30s, gracefulStop: 30s)
WARN[0160] The test has generated metrics with 100039 unique time series, which is higher than the suggested limit of 100000 and could cause high memory usage. Consider not using high-cardinality values like unique IDs as metric tags or, if you need them in the URL, use the name metric tag or URL grouping. See https://grafana.com/docs/k6/latest/using-k6/tags-and-groups/ for details. component=metrics-engine-ingester
WARN[0279] The test has generated metrics with 200011 unique time series, which is higher than the suggested limit of 100000 and could cause high memory usage. Consider not using high-cardinality values like unique IDs as metric tags or, if you need them in the URL, use the name metric tag or URL grouping. See https://grafana.com/docs/k6/latest/using-k6/tags-and-groups/ for details. component=metrics-engine-ingester
WARN[0513] The test has generated metrics with 400045 unique time series, which is higher than the suggested limit of 100000 and could cause high memory usage. Consider not using high-cardinality values like unique IDs as metric tags or, if you need them in the URL, use the name metric tag or URL grouping. See https://grafana.com/docs/k6/latest/using-k6/tags-and-groups/ for details. component=metrics-engine-ingester
█ setup
█ login
✓ login successful
█ search
✓ request successful
█ create
✓ request successful
█ delete
✓ request successful
checks.........................: 100.00% 518143 out of 518143
create_template................: avg=50.700907 min=15.053 med=39.333 max=903.808 p(90)=90.7192 p(95)=121.3456
data_received..................: 480 MB 498 kB/s
data_sent......................: 52 MB 54 kB/s
delete_template................: avg=40.795986 min=14.377 med=33.043 max=757.999 p(90)=66.4684 p(95)=85.858
get_detail_template............: avg=48.682064 min=16.633 med=30.133 max=2882.943 p(90)=70.2448 p(95)=125.2864
get_list.......................: avg=62.260015 min=16.032 med=36.691 max=2852.409 p(90)=108.0136 p(95)=172.5058
get_list_by_category...........: avg=60.343103 min=15.924 med=36.058 max=2809.757 p(90)=101.3828 p(95)=162.5264
get_list_by_tag................: avg=52.575703 min=14.211 med=30.645 max=2838.171 p(90)=83.3668 p(95)=141.447
group_duration.................: avg=105.69ms min=14.59ms med=51.25ms max=6.3s p(90)=197.92ms p(95)=306.68ms
http_req_blocked...............: avg=19.31µs min=0s med=0s max=159.12ms p(90)=1µs p(95)=1µs
http_req_connecting............: avg=5.86µs min=0s med=0s max=22.9ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=52.55ms min=14.21ms med=34.03ms max=2.88s p(90)=85.74ms p(95)=130.51ms
{ expected_response:true }...: avg=52.55ms min=14.21ms med=34.03ms max=2.88s p(90)=85.74ms p(95)=130.51ms
http_req_failed................: 0.00% 0 out of 518143
http_req_receiving.............: avg=220.07µs min=4µs med=51µs max=98.58ms p(90)=537µs p(95)=857µs
http_req_sending...............: avg=110.29µs min=7µs med=59µs max=25.26ms p(90)=169µs p(95)=292µs
http_req_tls_handshaking.......: avg=12.26µs min=0s med=0s max=70.4ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=52.22ms min=136µs med=33.72ms max=2.88s p(90)=85.42ms p(95)=129.98ms
http_reqs......................: 518143 537.150171/s
iteration_duration.............: avg=4.31s min=4.11s med=4.22s max=11.01s p(90)=4.52s p(95)=4.73s
iterations.....................: 86357 89.524856/s
vus............................: 61 min=5 max=400
vus_max........................: 400 min=400 max=400
running (16m04.6s), 000/400 VUs, 86357 complete and 0 interrupted iterations
default ✓ [======================================] 000/400 VUs 16m0s
성능 개선사항이 존재하지 않음. 기본값을 사용하도록 결정
k6 run group_test.js
/\ Grafana /‾‾/
/\ / \ |\ __ / /
/ \/ \ | |/ / / ‾‾\
/ \ | ( | (‾) |
/ __________ \ |_|\_\ \_____/
execution: local
script: group_test.js
output: -
scenarios: (100.00%) 1 scenario, 800 max VUs, 16m30s max duration (incl. graceful stop):
* default: Up to 800 looping VUs for 16m0s over 2 stages (gracefulRampDown: 30s, gracefulStop: 30s)
WARN[0185] The test has generated metrics with 100030 unique time series, which is higher than the suggested limit of 100000 and could cause high memory usage. Consider not using high-cardinality values like unique IDs as metric tags or, if you need them in the URL, use the name metric tag or URL grouping. See https://grafana.com/docs/k6/latest/using-k6/tags-and-groups/ for details. component=metrics-engine-ingester
WARN[0356] The test has generated metrics with 200002 unique time series, which is higher than the suggested limit of 100000 and could cause high memory usage. Consider not using high-cardinality values like unique IDs as metric tags or, if you need them in the URL, use the name metric tag or URL grouping. See https://grafana.com/docs/k6/latest/using-k6/tags-and-groups/ for details. component=metrics-engine-ingester
WARN[0695] The test has generated metrics with 400018 unique time series, which is higher than the suggested limit of 100000 and could cause high memory usage. Consider not using high-cardinality values like unique IDs as metric tags or, if you need them in the URL, use the name metric tag or URL grouping. See https://grafana.com/docs/k6/latest/using-k6/tags-and-groups/ for details. component=metrics-engine-ingester
█ setup
█ login
✓ login successful
█ search
✓ request successful
█ create
✓ request successful
█ delete
✓ request successful
checks.........................: 100.00% 378247 out of 378247
create_template................: avg=1057.126989 min=15.708 med=131.496 max=13894.01 p(90)=2288.838 p(95)=2390.012
data_received..................: 1.4 GB 1.5 MB/s
data_sent......................: 45 MB 46 kB/s
delete_template................: avg=1047.395176 min=15.173 med=120.948 max=13733.38 p(90)=2274.822 p(95)=2373.989
get_detail_template............: avg=1365.057512 min=17.144 med=235.184 max=14438.879 p(90)=2983.28 p(95)=3168.79
get_list.......................: avg=1470.42159 min=17.425 med=315.149 max=15191.771 p(90)=3134.052 p(95)=3412.904
get_list_by_category...........: avg=1466.581218 min=18.378 med=338.537 max=14916.961 p(90)=3146.44 p(95)=3433.486
get_list_by_tag................: avg=1459.650259 min=15.936 med=324.414 max=15248.148 p(90)=3154.238 p(95)=3442.665
group_duration.................: avg=2.62s min=15.43ms med=2.06s max=23.51s p(90)=7.37s p(95)=8.86s
http_req_blocked...............: avg=74.67µs min=0s med=0s max=486.54ms p(90)=1µs p(95)=1µs
http_req_connecting............: avg=34.95µs min=0s med=0s max=159.16ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=1.31s min=15.17ms med=270.86ms max=15.24s p(90)=2.96s p(95)=3.22s
{ expected_response:true }...: avg=1.31s min=15.17ms med=270.86ms max=15.24s p(90)=2.96s p(95)=3.22s
http_req_failed................: 0.00% 0 out of 378247
http_req_receiving.............: avg=298.98µs min=4µs med=72µs max=231.56ms p(90)=563µs p(95)=1.36ms
http_req_sending...............: avg=105.39µs min=7µs med=57µs max=28.03ms p(90)=167µs p(95)=297.69µs
http_req_tls_handshaking.......: avg=37.64µs min=0s med=0s max=191.46ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=1.31s min=623µs med=270.22ms max=15.24s p(90)=2.96s p(95)=3.22s
http_reqs......................: 378247 390.196249/s
iteration_duration.............: avg=11.86s min=4.12s med=11.93s max=31.93s p(90)=16.25s p(95)=17.46s
iterations.....................: 63041 65.032536/s
vus............................: 92 min=6 max=800
vus_max........................: 800 min=800 max=800
running (16m09.4s), 000/800 VUs, 63041 complete and 0 interrupted iterations
default ✓ [======================================] 000/800 VUs 16m0s
