In our last post we took a look at performance between async/sync handlers for tasks that are intrinsically synchronous. What about tasks that were built with async in mind? Typically these sort of tasks are things that involve IO, like file access, a db query, or a call out to a website. Does this influence performance?
IHttpAsyncHandler Code and Performance
Performance Results from 2000 requests at a concurrency level of 50
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.5 0 8
Processing: 275 2423 1352.5 2047 9957
Waiting: 259 2421 1353.1 2046 9957
Total: 276 2424 1352.5 2048 9957
Percentage of the requests served within a certain time (ms)
50% 2048
66% 2744
75% 3195
80% 3574
90% 4405
95% 4865
98% 5104
99% 6389
100% 9957 (longest request)
Performance Results from 3000 requests at a concurrency level of 50
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.5 0 4
Processing: 174 2992 1476.8 2879 12689
Waiting: 172 2991 1477.4 2878 12688
Total: 174 2993 1476.8 2880 12689
Percentage of the requests served within a certain time (ms)
50% 2880
66% 3605
75% 4042
80% 4245
90% 4790
95% 5059
98% 6352
99% 7524
100% 12689 (longest request)
Performance Results from 3000 requests at a concurrency level of 100
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.5 0 2
Processing: 273 5501 2835.9 5641 20909
Waiting: 270 5499 2836.7 5638 20908
Total: 273 5501 2835.9 5641 20910
Percentage of the requests served within a certain time (ms)
50% 5641
66% 6939
75% 7432
80% 7785
90% 8900
95% 10091
98% 11678
99% 12784
100% 20910 (longest request)
IHttpHandler Code and Performance
Performance Results from 2000 requests at a concurrency level of 50
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.5 0 2
Processing: 273 2419 856.3 1965 5861
Waiting: 271 2417 856.4 1964 5859
Total: 273 2420 856.3 1965 5862
Percentage of the requests served within a certain time (ms)
50% 1965
66% 2388
75% 2928
80% 3255
90% 3816
95% 4121
98% 4592
99% 4788
100% 5862 (longest request)
Performance Results from 3000 requests at a concurrency level of 50
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.6 0 7
Processing: 269 3608 3519.8 2080 23885
Waiting: 267 3606 3520.4 2078 23885
Total: 270 3608 3519.8 2080 23885
Percentage of the requests served within a certain time (ms)
50% 2080
66% 2535
75% 3155
80% 3850
90% 8223
95% 11518
98% 16501
99% 18591
100% 23885 (longest request)
Performance Results from 3000 requests at a concurrency level of 100
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.6 0 7
Processing: 494 5188 1228.9 4750 9665
Waiting: 492 5185 1228.9 4746 9662
Total: 494 5188 1228.9 4750 9666
Percentage of the requests served within a certain time (ms)
50% 4750
66% 5064
75% 5536
80% 5924
90% 6938
95% 7858
98% 8910
99% 9450
100% 9666 (longest request)
Bonus Tests! With IsReusable=false on both types
This surprsingly reversed the results we saw above, async was better--
IHttpAsyncHandler with 3000 requests, 100 concurrent, !IsReusable:
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.5 0 7
Processing: 199 6295 2689.3 6507 16925
Waiting: 196 6293 2690.2 6506 16925
Total: 199 6295 2689.3 6507 16925
Percentage of the requests served within a certain time (ms)
50% 6507
66% 7408
75% 7970
80% 8365
90% 9498
95% 10853
98% 12395
99% 13295
100% 16925 (longest request)
IHttpHandler with 3000 requests, 100 concurrent, !IsReusable:
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.6 0 12
Processing: 2833 10871 4946.1 11540 25468
Waiting: 2833 10870 4946.8 11540 25468
Total: 2833 10872 4946.1 11541 25468
Percentage of the requests served within a certain time (ms)
50% 11541
66% 12492
75% 13254
80% 13950
90% 17213
95% 19879
98% 21980
99% 23041
100% 25468 (longest request)
Analysis
Given this data, it appears as though IHttpAsyncHandlers seem to only pay off when you're designing a stateful handler (when you're using IsReusable=false), IHttpHandler conquers all other scnearios. Strange.
When a thread is given up there is some overhead associated with the context switch.
ReplyDeleteI wonder if the reason asynchronous isn't a standout winner is because the asychronous task you're doing is quicker than most; hitting google as opposed to interacting with a data store. Factor in the unusually quick async task with the context switch overhead and that might explain async's meager performance.
So what you're saying is that you'd like me to test the same thing with a DB? I can do that.
ReplyDeleteI'm starting to think that i need to post hitting google directly with all of these as well-- that way we can tell if any of the response times are actually "google lag" cascading back to my handler.