IHttpAsyncHandler vs IHttpHandler Performance, Take 2

In our last post we took a look at performance between async/sync handlers for tasks that are intrinsically synchronous. What about tasks that were built with async in mind? Typically these sort of tasks are things that involve IO, like file access, a db query, or a call out to a website. Does this influence performance?

IHttpAsyncHandler Code and Performance

Performance Results from 2000 requests at a concurrency level of 50

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.5      0       8
Processing:   275 2423 1352.5   2047    9957
Waiting:      259 2421 1353.1   2046    9957
Total:        276 2424 1352.5   2048    9957

Percentage of the requests served within a certain time (ms)
  50%   2048
  66%   2744
  75%   3195
  80%   3574
  90%   4405
  95%   4865
  98%   5104
  99%   6389
 100%   9957 (longest request)

Performance Results from 3000 requests at a concurrency level of 50

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.5      0       4
Processing:   174 2992 1476.8   2879   12689
Waiting:      172 2991 1477.4   2878   12688
Total:        174 2993 1476.8   2880   12689

Percentage of the requests served within a certain time (ms)
  50%   2880
  66%   3605
  75%   4042
  80%   4245
  90%   4790
  95%   5059
  98%   6352
  99%   7524
 100%  12689 (longest request)

Performance Results from 3000 requests at a concurrency level of 100

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.5      0       2
Processing:   273 5501 2835.9   5641   20909
Waiting:      270 5499 2836.7   5638   20908
Total:        273 5501 2835.9   5641   20910

Percentage of the requests served within a certain time (ms)
  50%   5641
  66%   6939
  75%   7432
  80%   7785
  90%   8900
  95%  10091
  98%  11678
  99%  12784
 100%  20910 (longest request)

IHttpHandler Code and Performance

Performance Results from 2000 requests at a concurrency level of 50

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.5      0       2
Processing:   273 2419 856.3   1965    5861
Waiting:      271 2417 856.4   1964    5859
Total:        273 2420 856.3   1965    5862

Percentage of the requests served within a certain time (ms)
  50%   1965
  66%   2388
  75%   2928
  80%   3255
  90%   3816
  95%   4121
  98%   4592
  99%   4788
 100%   5862 (longest request)

Performance Results from 3000 requests at a concurrency level of 50

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.6      0       7
Processing:   269 3608 3519.8   2080   23885
Waiting:      267 3606 3520.4   2078   23885
Total:        270 3608 3519.8   2080   23885

Percentage of the requests served within a certain time (ms)
  50%   2080
  66%   2535
  75%   3155
  80%   3850
  90%   8223
  95%  11518
  98%  16501
  99%  18591
 100%  23885 (longest request)

Performance Results from 3000 requests at a concurrency level of 100

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.6      0       7
Processing:   494 5188 1228.9   4750    9665
Waiting:      492 5185 1228.9   4746    9662
Total:        494 5188 1228.9   4750    9666

Percentage of the requests served within a certain time (ms)
  50%   4750
  66%   5064
  75%   5536
  80%   5924
  90%   6938
  95%   7858
  98%   8910
  99%   9450
 100%   9666 (longest request)

Bonus Tests! With IsReusable=false on both types

This surprsingly reversed the results we saw above, async was better--

IHttpAsyncHandler with 3000 requests, 100 concurrent, !IsReusable:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.5      0       7
Processing:   199 6295 2689.3   6507   16925
Waiting:      196 6293 2690.2   6506   16925
Total:        199 6295 2689.3   6507   16925

Percentage of the requests served within a certain time (ms)
  50%   6507
  66%   7408
  75%   7970
  80%   8365
  90%   9498
  95%  10853
  98%  12395
  99%  13295
 100%  16925 (longest request)

IHttpHandler with 3000 requests, 100 concurrent, !IsReusable:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.6      0      12
Processing:  2833 10871 4946.1  11540   25468
Waiting:     2833 10870 4946.8  11540   25468
Total:       2833 10872 4946.1  11541   25468

Percentage of the requests served within a certain time (ms)
  50%  11541
  66%  12492
  75%  13254
  80%  13950
  90%  17213
  95%  19879
  98%  21980
  99%  23041
 100%  25468 (longest request)

Analysis

Given this data, it appears as though IHttpAsyncHandlers seem to only pay off when you're designing a stateful handler (when you're using IsReusable=false), IHttpHandler conquers all other scnearios. Strange.

IHttpAsyncHandler vs IHttpHandler Performance

Got into a conversation today with my brother (and tech-blog editor extraordinaire) Justin Smith, about the performance of async vs non-async handlers/pages. Thought it'd be interesting to test out if you only reap the benefits of async handlers/pages when you're passing work off to a webservice or db. So, we're going to test throughput/response time of asnyc vs synchronous Fibonacci calculations.

Note all tests were ran on my localhost using in debug="false" mode, which is very important as with it on test results are completely different. Here are the tests and results:

IHttpAsyncHandler Example and Performance

The TestMethods class at the top was used in both tests.

Performance results for 2000 requests at a concurrency level of 50:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       1
Processing:     6   31   6.4     30      60
Waiting:        5   31   6.3     30      60
Total:          6   31   6.4     31      60

Percentage of the requests served within a certain time (ms)
  50%     31
  66%     32
  75%     35
  80%     36
  90%     39
  95%     42
  98%     47
  99%     48
 100%     60 (longest request)

Performance results for 5000 requests at a concurrency level of 50:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       2
Processing:     5   29   5.7     30      81
Waiting:        5   29   5.7     29      79
Total:          5   30   5.7     30      81

Percentage of the requests served within a certain time (ms)
  50%     30
  66%     31
  75%     33
  80%     33
  90%     37
  95%     39
  98%     42
  99%     44
 100%     81 (longest request)

Performance results for 5000 requests at a concurrency level of 100:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       4
Processing:    12   61  11.1     62      88
Waiting:       12   61  11.1     62      87
Total:         12   62  11.1     62      88

Percentage of the requests served within a certain time (ms)
  50%     62
  66%     66
  75%     68
  80%     69
  90%     75
  95%     80
  98%     83
  99%     85
 100%     88 (longest request)

IHttpHandler Example and Performance

Performance results for 2000 requests at a concurrency level of 50:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       2
Processing:     6   23   4.6     23      55
Waiting:        6   22   4.6     22      55
Total:          6   23   4.6     23      55

Percentage of the requests served within a certain time (ms)
  50%     23
  66%     24
  75%     25
  80%     26
  90%     28
  95%     30
  98%     34
  99%     37
 100%     55 (longest request)

Performance results for 5000 requests at a concurrency level of 50:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       4
Processing:     8   23   4.3     22      52
Waiting:        7   22   4.2     22      52
Total:          8   23   4.3     22      52

Percentage of the requests served within a certain time (ms)
  50%     22
  66%     24
  75%     24
  80%     25
  90%     27
  95%     30
  98%     36
  99%     40
 100%     52 (longest request)

Performance results for 5000 requests at a concurrency level of 100:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       5
Processing:     7   47   6.9     45     138
Waiting:        7   46   6.8     45     114
Total:          7   47   6.9     45     138

Percentage of the requests served within a certain time (ms)
  50%     45
  66%     48
  75%     50
  80%     51
  90%     55
  95%     59
  98%     66
  99%     68
 100%    138 (longest request)

Analysis

So synchronous seems like the way to go-- no real gains on async. The next tests will be for IO bound tasks (web/db calls).

.NET Rx Driven Web Server, Take 2

Feeling like my code was rather crappy, i thought i'd take another stab at the Rx Webserver problem. I was especially unhappy with the async signature of Socket.BeginRecieve. A little research online and i found HttpListener, a .NET class that does the same thing except with a usable Async function (one that returns something meaningful) for my Observeable.FromAsyncPattern.

A Hyper-fast Rx Web Server

This one actually performs! Here's the code:

Edit: I found this after making mine-- José F. Romaniello's implentation of a .NET Rx Web Server. His code looks much more useful, and is a better example of Rx. My version hardly Rx, really. Anyways, i performance tested his as well and the figures were similar to the tests ran on this one below.

Performance Tests-- MVC3, Nodejs, and Our Rx Server

MVC3

Using a generic controller that returns "Thanks!":

Server Software:		ASP.NET
Server Hostname:		localhost
Server Port:		31362
Document Path:		/Test/
Document Length:		11 bytes
Concurrency Level:		1000
Time taken for tests:		42.845 seconds
Complete requests:		10000
Failed requests:		0
Total transferred:		2700000 bytes
HTML transferred:		110000 bytes
Requests per second:		233397.92
Transfer rate:		63017.44 kb/s received
Connnection Times (ms)
	min	avg	max
Connect:	0	0	590
Processing:	688	4077	4649
Total:	688	4077	5239

Our server

Using the code above:

Server Software:		Microsoft-HTTPAPI/2.0
Server Hostname:		localhost
Server Port:		8081
Document Path:		/Test/
Document Length:		7 bytes
Concurrency Level:		1000
Time taken for tests:		1.624 seconds
Complete requests:		10000
Failed requests:		0
Total transferred:		1500000 bytes
HTML transferred:		70000 bytes
Requests per second:		6157021.28
Transfer rate:		923553.19 kb/s received
Connnection Times (ms)
	min	avg	max
Connect:	0	0	2
Processing:	39	155	180
Total:	39	155	182

Node.js

Using the node.js homepage hello world sample:

Server Software:
Server Hostname:		localhost
Server Port:		1337
Document Path:		/
Document Length:		8 bytes
Concurrency Level:		1000
Time taken for tests:		2.066 seconds
Complete requests:		10000
Failed requests:		0
Total transferred:		720000 bytes
HTML transferred:		80000 bytes
Requests per second:		4839481.65
Transfer rate:		348442.68 kb/s received
Connnection Times (ms)
	min	avg	max
Connect:	0	7	94
Processing:	1	35	244
Total:	1	42	338

The Actual Node.js Beater

Really, there's a good bit of apples to oranges here, but still--can you believe that? I think that means that our little Rx server is C10K compliant! Further, many of our server's metrics beat node.js-- such as transfer rate, requests per second, max times... pretty awesome!

I've still got a ton to learn about Rx... back to the books now.

A .NET Rx Driven Web Server

Edit: please see my other post on creating a .NET Web Server from Rx (Reactive Extensions) since it contains better code.

Although I've seen .Net Rx (Reactive Extensions) around, I never messed with them until today. To me, the concepts behind Rx always seemed self explanatory--perhaps because i have accomplished concurrent apps in .NET 1.0/2.0 without them. However, having spent a little time with them today, I think Rx is good stuff. Honestly, I'm impressed with the interfaces and the services they provide. Let's check it out:

What are the .Net Reactive Extensions(Rx)?

Short answer: pubsub.

Long answer: a ton of sugar over top of .Net streams, async, TPL, and pubsub. I'm not going to get into the generic intros you can find elsewhere that involve streaming enumerables to the console. Instead I'd prefer to create the argument for Rx as such-- when given the need for "X", it is better to provide "the ability to provide X" than "X" itself. The Reactive Extensions give you a ton of really helpful methods to aide you in implementing "the ability to create X" over "X" itself. Allow me to explain--

If i asked you to write me a function that gave me the first 1 million numbers, how would you implement it? I know a younger me would've started on cranking out a for loop, not taking into consideration that decision's implications upon the system's memory. A smarter implementation would be to give me a function/object that gives me the ability to create the first million numbers, perhaps through iterating through the set. Such an object could then forgo the previously mentioned memory issues. The idea of giving "the ability to create/observe X" over "X" itself is arguably the conceptual basis of functional programming's lazy evaluation, which is also what Rx aims to help the user create (to me, at least). So, out of the box you get a ton of ways to create and enable the push/pull of streaming data and/or events.

An Rx TCP Server

The first thing i could think of to make with Rx is a single-threaded TCP server. Maybe that's because when i think of streaming data these days, i tend to think of a node.js style web server. How hard could it be? (And what would the performance be like...?

Version One: A Single-Threaded Non-Rx Readonly TCP Server

If you run the following code, and make a request on your web browser to http://localhost:8081 you'll see the GET request come through to the app.

Version Two: A Single-Threaded Rx Enabled TCP Server

In this version I added two properties to the NetActor-- Incoming and Outgoing. Both are based on new Rx interfaces that allow the client to tap into the push/pull of data to the client. So if you open your web browser, open up the localhost site, and then type into the console app and press enter, it will get delivered to the web page:

Version Three: The Node.js Killer

Ok, so in order to get apache bench to recognize my console app as a web server i had to bind the NetActor's Ip to something other than localhost... not sure why. Once i got that working, i had intermittent failure until I implemented part of the HTTP spec-- at least the response code and connection closed. After that, and also after creating the ability for the NetActor to shut itself down and start itself up, here is what i was left with:

Apache Bench Results

At 500ms+ with a concurrency level of 1, this is definitely not a node.js killer..... ;-)

node.js http request example

A http server acting as a google reverse geocoder API proxy.

You just need to pass in long/lat as querystrings, for example, "localhost/?hi=there&long=37&lat=37". I put the hi=there in there because node querystring module parses strangely...

Why Functional Programming [Still] Matters

With some of my break time i've been reading the paper Why Functional Programming Matters by John Hughes. It's a short paper, weighing in at only twenty-two pages in length. However, after hours of reading i'm only eleven pages in. Although i haven't finished it, i want to sum up my thoughts on it so far in 2012-speak. The original paper was written in 1984 and used Lisp for all the examples, which is sort of an obsolete vernacular for today's web-based programmers. All and all, i thought i'd attempt breathe some life into this classic and write up a bit of it using javascript instead.

So here goes, Why Functional Programming Still Matters in 2012:

Introduction

In the introduction John stated a few facts that blew my mind:

• Functional programs contain no assignment statements. Variables are given a value once, and then they never change. (otherwise known as immutability)
• Functional programs contain no side-effects at all (and less bugs), since "a function call can have no effect other than to compute its result." Reminds me of a chapter i just finished reading in DDD
• Expressions can be freely replaced with variables and (more importantly in my opinion) vice versa. It's so important that I want to write it out-- variables can be replaced by expressions
• Functional programs are more modular, and functionality is easier to swap
• Programmers who take the functional program route are an order of magnitude faster than ones that do not

At this point you should want to find out why functional programming is the greatest thing in the world. At least that's where i was at mentally at this point. I don't mean to spoil it for you, but as we read we'll find out that the key to functional programming's greatness is its modularity.

An Analogy with Structured Programming

John unofficially defines a structured program as something that allows for modularity. Anyone who has done anything even remotely serious in programming knows how important modularity is. I really like John's quote on the matter:

When writing a modular program to solve a problem, one first divides the problem into sub-problems, then solves the sub-problems and combines the solutions. The ways in which one can divide up the original problem depend directly on the ways in which one can glue solutions together. Therefore, to increase ones ability to modularize a problem conceptually, one must provide new kinds of glue in the programming language.

I think all of us can speak to that in one shape or another-- we create functions that plug in to other functions that plug into the main execution of the program and it's all reusable and all that-- but the point John is trying to make is that the difference between functional programs and others is where you're able to be modular.

So in other words, the argument is that "functional programming has the best glue." Let's start taking a look at the sort of glue John would have us use.

Interlude-- A Bit of Lisp

Before we get there we need to bring the conversation into the present. IMO John has a little something going for him and his argument--Lisp. Yes, although Lisp is years old it's really advantageous for functional programming. Why? To start with, because its lists are functions. (This is all in my opinion, please note that i am not a Lisp-master and only briefly messed around with languages like Lisp and MIT Scheme in college @ Wheeling Jes)

I know you need that explained. In Lisp, lists are created using the function cons. So what we call [1,2] in JavaScript is (cons 1 (cons 2 nil)) in Lisp. Simply put, cons means "make the two things after me into a list." The nil you see there is null in Lisp, which essentially in this example means "end of the list." So the second half (cons 2 nil) means "make 2 and nothing else into a list." Lisp is also written in prefix notation, meaning that 2+2 in JavaScript is (+ 2 2) in Lisp--the function/operator comes first. It looks wierd but you could get used to it.

The 2+2 code is just another advantage Lisp has-- it lacks JavaScripts syntactical impedence between functions and operators.

All and all, there's a ton of swapability baked into that. Again, let's use the 2+2 example. If i wanted to change the JavaScript one to instead call a function we made called Superify(x,y) on it that does more than just add, we would have to convert 2+2 to Superify(2,2). In Lisp, we would only have to swap all instances of the + operator to make (Superify 1(Superify 2 nil)) and we're all done.

And there's more that we'll get into later... back to the paper.

Glueing Functions Together

If i asked the majority of programmers to make a function that adds up items in a list, i'd probably get something like so:

If i then asked them to create a function that multiplies all the items in a list, i'd probably get someting like so:

A lot of duplicate code between those two, and not a ton of modularity-- take a look. Both are iterating through a list, doing something with the item, and then returning the result. Further, each has its own default/start value in the form of sum--notice how its defaulted to 1 for multiplication and 0 for sum?

John discussed this, suggesting instead that we make the code more modular by creating a function that allows us to keep the similarities, and pass in the differences. In his paper, he suggested a function called reduce with the form reduce(func, list, a), where func is a function to apply to each item, list is a list/array of objects, and a is what to substitute in instead of nil, or what to use as a default.

Here's what this may look like in JavaScript:

Look at how modular that is-- we can create new functions that work with a list of items and easily get them up and running without rewriting all of that boilerplate iteration code. Hughes calls functions like reduce "higher order functons."

John also describes a function that walks over a list, applying a function to each item in the list and returning a new list containing the results--map. Here is an example of map written in JS.

So let's say we wanted to print out all the items in an array, we could do that pretty simply with the map function like so: map(alert,anArray) and skip all the boilerplate for-each-ing.

Where to Go From Here

While I may at a later date write up more about the rest of the paper (such as what lazy evaluation is) and how to achieve it in JavaScript, i'm pretty happy with the time i've spent so far on this.

One more small piece of advice-- don't write your own map/reduce. Use something like underscore.js that is as browser-optimized as possible.