Three Web-servers

DeepGeek

December 2009

Abstract: My impressions of testing three different web-servers at the home lab.

1 Introduction

I recently tested three different web-servers at the home lab. Let me preface this by saying that I am not trying to offer “professional” advice. This is just to give my impressions of seeing what it is like to compare different web-servers in a home lab.

2 Why?

Why would somebody test out different web-servers in a home? Curiosity is the answer. And specifically curiosity about some of the servers out there that aren’t Apache. This is not to knock Apache in some way, it is a fantastic server. But there has to be reasons for there to be other web-servers. One difference is that I was interested in a particular hybrid web & gopher server. The other was that I have a thing for lightweight applications, and some web servers feature themselves as being lightweight.

3 Will the applicants please stand up?

The servers I tested were “pygopherd,” “nginx,” and “thttpd.” I wanted to also test the fnord web-server, but it required a TCP server, which appeared to be a replacement for inetd, as well as recommending a different system initialization system. Interesting requirements and recommendations, but not something I was willing to take on at the moment.

Recently I did an episode (episode # Hex 14,) where I discussed the Gopher protocol. One of the gopher servers was “pygopherd.” This server has an interesting set of features. First, even though it is listening on the port for the gopher protocol, it will detect a few other protocols, specifically web and wap, which is a mobile web protocol. Pygopherd will then convert the gopher pages to HTML or wap on the fly for these devices.

A second one I tried out was “nginx.” Nginx is a project of Russian origin, and it is a lightweight performance webserver.

Lastly, I also tried “thttpd.” The “t” in the first digit stands for “tiny/throttling/turbo.”

4 Setting up the Test Bed

So I figured that I wanted to test on a separate computer, because it would allow me to leave the server testbed running and independent of my main computer. I called up a buddy from a Linux SIG and asked him if he had a spare computer. Surprisingly enough, he offered two, “one is more powerful, the other has a smaller case.” I asked about quiet, as I planned to have the thing running around the clock in my bedroom, and he gave me the smaller one. It was a Celeron with small memory. I loaded Debian Lenny and installed pygopherd, nginx, and thttpd.

I had this expectation that a few friends on the net would be interested in trying out the gopher server with me, and set up accounts, and set up port forwarding from my DSL modem back to this computer. While this interest was expressed, my net-friends did not actually do much by the way of trying things out on it. I can only imagine that the interest would really be there if I had decided to set up a proper gopher server an a VPS with the attendant conveniences, but that is speculation.

5 Pygopherd

I mentioned the big feature of Pygopherd - multi protocols. This server is actually written in Python, and so requires Python to be running. This server was a cinch to install, I did not have to touch the configuration file, the defaults were sensible.

I quickly set up a test gopher site, and checked it out from my desktop. Everything looked fully compliant. Response was incredible, but this was an impression, not a measurable thing. I then hit it with the Firefox web browser, and it gave me the same site, with a stanza at the top to inform me that I was viewing a gopher site through a web site. I also checked it out on my cell phone, again, that web browser look but on the small screen, and the software was sensible enough not to tell me I was using a WAP version of a gopher page.

I did notice that the configuration file had provisions for some other features worth mentioning. One feature was to store the files in a compressed Zip file, and have the server pull what it needs directly out of the archive. Another was to serve up emails received in the standard *nix mailbox formats as gopher pages. Even another feature was to switch from a standard forking server to a multi-thread server.

The server is very interesting to run, depending on how I set up the gopher menu, it will either use the gopher server to download a file or refer the gopher browser to a HTTP style download. I really enjoyed the flexibility.

6 Nginx

Nginx is a web server that competes with Apache in feature set, but is billed as being more secure and faster.

Now the faster is an interesting thing, I briefly mentioned in the Pygopherd section that pygopherd can switch between standard forking and a multi threaded mode. Nginx earned it’s reputation for speed by being an early adopter of the multi-thread technique, which is faster on systems with a large number of cores as well as systems that serve bigger files. Since then, many servers offer a choice of how the server behaves under the hood.

Another aspect of Nginx is it’s ability to send web pages in gzip form between itself and the browsers that attach to it. I was really unaware that a change took place in web browsers, and the biggest, AFAIK Apache, nginx, Lighttpd, and the uber light Fnord I mentioned earlier, can take HTML and TXT and use the GZIP and Deflate algorithms to compress a file in transit. Upon finding this out, I checked out a variety of web browsers in the Debian repository and was surprised to find that they all linked to the same decompression library. I believe, that is to say it is speculation, that this change was originally about speed, but I think that now with the higher availability of broadband, that it is more about bandwidth savings now.

I must say that even though nginx was fast, that I found the configuration system, a stanza-like configuration like Apache, more difficult to set up. It required that I used documentation that was not installed in a manpage on my server, but instead I found myself navigating a large wiki with sparse examples. Now I did mention that I was testing this at a home lab, and I don’t want to get on Verizon’s “bad” list, so I wanted to use alternate ports in case they were monitoring for traffic based on ports used. I never did manage to get nginx to stop listening to port 80, but I could assign a non-standard port to it, and even point web traffic from that port to a specific directory.

7 Thttpd

We all know that the “httpd” is for “HTTP daemon.” The “T” is for Tiny/Turbo/throttling. This is a small executable web server that is not full featured like nginx, but maybe has what is needed for more static content. As matter of fact, some big web servers will use this server to serve static pages by setting up this server to be a helper application for Apache, and thus allow Apache to save it’s big resources for more sophisticated Ajaxy pages.

So, it is not featureful, but it is very simple to set up. It has two configuration files, one of which is for setting throttling features. Thus, with this server, you can give the whole server limit on bandwidth it may use, as well giving specific groups of URLs (as defined via wildcard-style declarations,) bandwidth limits. The regular configuration file is a simple, flat list of items each with comments explaining the setting, the default, and then you can override as you need to. Of if you want, you can override the configuration file with command line switches if you like. This server, like nginx but unlike pygopherd can create Apache style logs that can be interpreted by web page statistic analyzer packages.

8 Siege

The time came to benchmark the servers with a tool called “Siege,” as in “the castle is under siege.” Siege creates “drone web surfers” in memory and then all these subprograms ask for pages repeatably, marking statistics for the web pages.

While Siege can be used for nefarious purposes, I enjoyed using it to test these servers over my 100 MB/s LAN. Both Nginx and Thttpd successfully served 600-700 text pages per minute. Nginx a little more than Thttpd.

Pygopherd was another story, it served in the hundreds. Since Pygopherd allowed me to switch modes from threaded to forked, I actually noted that forked was less efficient. On top of that, while in forked mode I could actually generate situations in Siege where Siege would make the server fail. This proved to my satisfaction that indeed, the reason gopher servers seemed so much faster on the web was because they just aren’t used as much. This is not to knock gopher servers, and using gopher may actually be a good way for the computer savvy to a little more a little faster out of the net.

Some may have noted that I said that the test traffic generated by Siege did cause a successful DOS attack against pygopherd, and I want to tell my listeners, please don’t abuse Siege, that’s keep this an honest tool. But I did manage to find an interesting application for it. As you may recall, early in the series I talked about choosing a web-host, and probably will use Siege as a tool for this if I ever go through this process again, let me tell you why. Now I have a “non-oversold” web-hosting account, but two of my net-acquaintances don’t, one uses Dreamhost’s unlimited, and the other uses banner sponsored free web-hosting from godaddy. Hey, we all have different ideas on the right host for our needs. But just for giggles, I pointed Siege at these three servers for thirty seconds and thirty simulated users retrieving text and HTML. Well, I am happy to say that my service was the best. Second, and very close, was Dreamhost. Considering their overselling nature, I interpret this as a testimony to the way their techs set up their servers. Last was the godaddy pages. So how many neighbors you have sharing your webserver’s resources does count when it comes to performance. A little confirmation is nice, you know? I wonder what the reaction would be if you called the salesmen of several web-hosts, and then asked them for sample URL’s and reported their servers responses back to them as part of your shopping process. One can wonder, huh?

Talk Geek To Me

Internet Radio For Computer Geeks