Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark matrix for connect protocols #588

Merged
merged 3 commits into from
Oct 23, 2023

Conversation

emcfarlane
Copy link
Contributor

Previous benchmarks only tested gRPC clients. This extends the matrix for all protocols and each streaming type. Streams do a single req/rsp to make it easier to compare to unary flows.

go test -run=^$ -bench=BenchmarkConnect
goos: darwin                                        
goarch: arm64                                                     
pkg: connectrpc.com/connect                                      
BenchmarkConnect/connect/unary_big-8                 517           1978663 ns/op         5784987 B/op        171 allocs/op
BenchmarkConnect/connect/unary_small-8             29280             40443 ns/op           21701 B/op        165 allocs/op
BenchmarkConnect/connect/client_stream-8           20979             57211 ns/op           22458 B/op        206 allocs/op
BenchmarkConnect/connect/server_stream-8           20085             56513 ns/op           22721 B/op        211 allocs/op
BenchmarkConnect/connect/bidi_stream-8             20209             58290 ns/op           21546 B/op        188 allocs/op
BenchmarkConnect/grpc/unary_big-8                    537           2037258 ns/op         5940546 B/op        231 allocs/op
BenchmarkConnect/grpc/unary_small-8                25891             46011 ns/op           24805 B/op        223 allocs/op
BenchmarkConnect/grpc/client_stream-8              25942             45948 ns/op           26083 B/op        228 allocs/op
BenchmarkConnect/grpc/server_stream-8              26628             45404 ns/op           26277 B/op        234 allocs/op
BenchmarkConnect/grpc/bidi_stream-8                25088             48019 ns/op           21529 B/op        211 allocs/op
BenchmarkConnect/grpcweb/unary_big-8                 541           2059821 ns/op         6236843 B/op        218 allocs/op
BenchmarkConnect/grpcweb/unary_small-8             20353             57924 ns/op           30645 B/op        211 allocs/op
BenchmarkConnect/grpcweb/client_stream-8           20475             59153 ns/op           30904 B/op        216 allocs/op
BenchmarkConnect/grpcweb/server_stream-8           20637             57831 ns/op           31755 B/op        221 allocs/op
BenchmarkConnect/grpcweb/bidi_stream-8             20388             58601 ns/op           20090 B/op        189 allocs/op          
PASS                                                              
ok      connectrpc.com/connect  24.837s   

@emcfarlane emcfarlane self-assigned this Sep 12, 2023
bench_test.go Show resolved Hide resolved
bench_test.go Show resolved Hide resolved
bench_test.go Outdated Show resolved Hide resolved
handler_example_test.go Outdated Show resolved Hide resolved
@akshayjshah
Copy link
Member

Not super surprised, but it's nice to see that the Connect protocol is the most efficient unary implementation.

@emcfarlane
Copy link
Contributor Author

@akshayjshah let me know what else is needed for this PR. Would be great to have for testing the changes to unary flows!

Copy link
Member

@akshayjshah akshayjshah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I wasn't sure about expanding the example PingServer implementation, but it's probably good to show how to implement all the different RPC types in the GoDoc.

@akshayjshah akshayjshah merged commit 0ad8df0 into connectrpc:main Oct 23, 2023
7 checks passed
@akshayjshah
Copy link
Member

Updated the commit description before merge. For posterity:

$ BENCH=BenchmarkConnect make bench

BenchmarkConnect/connect/unary_big-12         	   1041	  1183182 ns/op	6379285 B/op	    175 allocs/op
BenchmarkConnect/connect/unary_small-12       	  25263	    46785 ns/op	  19597 B/op	    151 allocs/op
BenchmarkConnect/connect/client_stream-12     	  17299	    69526 ns/op	  26505 B/op	    189 allocs/op
BenchmarkConnect/connect/server_stream-12     	  17530	    69564 ns/op	  26562 B/op	    184 allocs/op
BenchmarkConnect/connect/bidi_stream-12       	  17264	    68655 ns/op	  22356 B/op	    172 allocs/op
BenchmarkConnect/grpc/unary_big-12            	    867	  1193301 ns/op	6283560 B/op	    234 allocs/op
BenchmarkConnect/grpc/unary_small-12          	  20544	    57809 ns/op	  23892 B/op	    209 allocs/op
BenchmarkConnect/grpc/client_stream-12        	  19783	    59283 ns/op	  24501 B/op	    212 allocs/op
BenchmarkConnect/grpc/server_stream-12        	  20704	    57771 ns/op	  24781 B/op	    206 allocs/op
BenchmarkConnect/grpc/bidi_stream-12          	  19772	    57777 ns/op	  20032 B/op	    195 allocs/op
BenchmarkConnect/grpcweb/unary_big-12         	    859	  1215487 ns/op	6008795 B/op	    220 allocs/op
BenchmarkConnect/grpcweb/unary_small-12       	  17203	    68704 ns/op	  30543 B/op	    196 allocs/op
BenchmarkConnect/grpcweb/client_stream-12     	  17422	    68824 ns/op	  30042 B/op	    199 allocs/op
BenchmarkConnect/grpcweb/server_stream-12     	  17274	    68969 ns/op	  30180 B/op	    193 allocs/op
BenchmarkConnect/grpcweb/bidi_stream-12       	  15910	    74695 ns/op	  21374 B/op	    172 allocs/op

@emcfarlane emcfarlane deleted the ed/bench branch October 23, 2023 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants