Tomcat 6.0.18 vs Jetty 6.1.14

When I started programming servlets some years ago, Tomcat (5) was the most natural choice to choose as servlet container. It wasn't just the best known container, but it was in addition blessed by SUN as the reference implementation, so it always had the latest compatible features. In addition, Tomcat came bundled with my IDE, so I didn't even think about using a different container. Over the years, the situation has changed a bit. SUN moved to Glassfish/Grisly as reference implementation and I got the impression, that the hype moved from Tomcat to Jetty as the preferred container.

Being lazy, I wouldn't have thought about switching if it wasn't due to some problems I had when I had to upgrade from Tomcat 6.0.14 to 6.0.18. Suddenly, my jsp contained all errors and my cookies threw errors. Both problems were conflicts with the specs: I used 'empty(var)' in my code instead of 'empty (var)' (watch the space) and I didn't uuencode the cookies. But I don't want to see such problems turn up when I just patch because of a security advisory.

Another reason to try something else was the memory consumption. Memory is cheap, but when running servlets on a virtual machine from a hosting provider, price increases close to linearly with memory, since cpu power usually is no issue for people sharing servers.

I decided then to give Jetty a try, since it is known to be small in size and well-suited for embedded applications, which usually have memory limits. My first impression wasn't very good. Download size is quite big, I didn't manage to compile jetty by myself and I couldn't find good documentation. But after some try and error sessions, I managed to get jetty running. The missing documentation turned out to be my biggest problem for setting up all the features I needed for my application, namely: virtual hosts, JNDI database pool, compressed html pages. Searching for documentation on the web didn't help so much since the configuration seems to have changed between Jetty 4, 5 and 6. Chances to get a wrong hint are higher than getting a correct one. The best source of documentation are the javadocs. Here it is clear, that Jetty is more used as embedded container, which needs to be configured on the java level, rather than a standalone application. The configuration file is rather java translated to xml than user-friendly. There were some other inconveniences, but all could be solved.

Finally everything was set up, and I could start testing both containers for performance, and more importantly, for memory consumption. I used the flags which I found out worked best for my application
on tomcat on a two-processor machine:
JAVA_OPTS='-Xmx96m -Xms96m -Djava.awt.headless=true -XX:+UseConcMarkSweepGC -XX:PermSize=24m -XX:MaxPermSize=24m'
and I tested with:
ab -c 20 -n 10000 'http://localhost:8080/dict?searchItem=warum'
I measured the first, the second, the third and the forth 10000 requests, total memory consumption was measured with top-RES.

	#/s 10000	#/s 20000	#/s 30000	#/s 40000	Memory consumption (30000)
jetty -server	224	391	511	512	148m
tomcat -server	248	525	639	33	170m
jetty -client	383	408	414	414	78m
tomcat -client	496	564	543	46	126m

The results show that tomcat memory consumption is more than 20% larger than jetty, while using a client-vm gives another 20%. For performance, the opposite is approximately the case. In addition, after about 35000 requests, tomcat performance drops dramatically, and analysing this further, this is caused mainly by the GC working permanently. I haven't seen that during normal load, where I have approx. 70000 hits a day, so I think this must be some kind of session data, which is kept in tomcat until the session times out after 30min. Talking about real-time usage: I have now jetty running for some days, and I don't recognize any differences. Performance is good, and memory-consumption approximately the same as before. Maybe the most important lesson I have learned: Run with client-vm when running low on memory.