For some reason, running an integration test against your web application or web services is never really awarding. I can come up with some reasons, but there are probably more, because I always find myself trying to avoid writing them, which is a bad habit. Here are some of the reasons why I don't like them:
- Configuring the application server correctly for your purposes is just way too complicated
- Starting the web server just takes way too much time.
- Making sure that your build does not claim a port already in use by something else is just way too much work.
- Setting up your sniffer to check what is actually going across the wire is way too much work
- Having to dig through the application server's log files is just too much of a hassle
Now, there is something called Cargo, which was going to make life a lot easier. But it seems the project is no longer active, and I never really found it easy to use.
Last week, I wanted to test if my message authentication scheme was working correctly. So I needed to check if the client correctly signed a message sent to a web service, and then check if the server would correctly grant or deny access to that service. I resisted the temptation of going for Cargo, and had a look at some of the alternatives.
Embedding Jetty
I never really looked into the APIs for embedding Jetty inside of your process, but it is EASY. It's just perfect. Running Jetty from within your tests is hardly any work at all. This is about it:
server = new Server();
SelectChannelConnector remoteConnector = new SelectChannelConnector();
remoteConnector.setHost("127.0.0.1");
remoteConnector.setPort(8090);
server.setConnectors(new Connector[] { remoteConnector });
WebAppContext context = new WebAppContext();
context.setContextPath("/");
context.setWar(getPath("src/main/webapp"));
context.setClassLoader(this.getClass().getClassLoader());
server.setHandler(context);
server.start();
Most of this is Jetty API. The only exception is the static
getPath() operation, which is expected to return the absolute path
for the relative path passed in. (Always guaranteed relative to the
Maven module's base directory, regardless of the execution
environment. So it returns the same absolute path both in Maven as
well as Eclipse.)
Embedding Jetty with a LocalConnector
So this works quite ok. By just adding a TestSetup, it would be possible to start and stop the server for a whole slew of tests. Pretty cool. But then - why do we actually need a connector listening on a socket? The client that is going to make the request is the test itself. It doesn't seem all that useful to set up a socket listener.
And it turns out that you don't have to do that. Instead of passing a SelectChannelConnector instance, you can actually also pass a LocalConnector. And the LocalConnector has an operation that expects the request as a String (or a buffer of bytes), and returns a String with the response (or a buffer of bytes). That way, you basically cut out the entire network transport. The only thing you need to do is make sure that your TestCase retains a reference to that LocalConnector, so that you it use it in each and every test.
localConnector = new LocalConnector();
server.setConnectors(new Connector[] { localConnector });
WebAppContext context = new WebAppContext();
context.setContextPath("/");
context.setWar(getPath("src/main/webapp"));
context.setClassLoader(this.getClass().getClassLoader());
server.setHandler(context);
server.start();
Now, that's all pretty sweet. However, my service is based on Hessian. And I want to use the normal HessianProxyFactory and other Spring Hessian client code. And unfortunately, the Hessian classes expect a URL. And there is no protocol in the VM that allows me to send an HTTP request to non-socket listener. What can we do about it?
Loopback URLConnection
I knew that one of the ways to solve it would be to write my own URLStreamHandler for my own protocol specifier, and then have my own URLConnection subclass that would call the LocalConnector mentioned before. But it's not easy. First of all, no matter what base class you start of with, your URLConnection class will have to create the HTTP messages itself. And parse the HTTP response itself. The getOutputStream() and getInputStream() operation deal with the content only. And whatever I would do, it needed to be compatible with the normal way of interaction with a URLConnection and HttpURLConnection. And as it happens, the contract of HttpURLConnection and URLConnection is ridiculously ill-defined. Taking a look at the JavaDocs doesn't help. In fact, the documentation is sometimes just plain wrong.
But no matter what I tried, there was no way to work around that. If I wanted to use the Spring abstractions, then it would involve having a HessianProxyFactory, the endpoint would be identified with a URL, and the connection would be set up by opening a connection based on that URL.
So I sat down and created a first version. At first, I was writing my own HTTP requests, making sure the encoding was done properly, making sure the Content-Length matched the actual size of the content section, and so on and so forth. However, at some point, I realized it was getting way to complicated, so I called in help from commons-httpclient.
It took me some time before I got a grip on the API from commons-httpclient. I don't think the creators originally had something in mind like the stuff I wanted to do. I refactored my existing solution for a while, and then realized this wasn't going to cut it either.
And then I turned back to Jetty. It turns out Jetty has a very nice HttpGenerator and HttpParser. I liked the abstractions, but what I liked even better is that the HttpTester class basically formed a convenience wrapper around these classes. So I decided to start with HttpTester as the base class of my own HttpURLConnection subclass.
As it turns out, HttpUrlConnection has a lot of operation that all need to behave slightly differently depending on the state of the request. So, you cannot for instance change the request properties after the request has been sent. Having to deal with that in every method was just way more than I could bear. So I decided to make my own HttpURLConnection a simple facade to a strategy with implementation of these methods, and made sure that the strategy was replaced on state changes. Finding what to do in every state turns out even more complicated then implementing it, and in many cases I had to rely on bug reports to figure out how HttpURLConnection is actually implemented. (And how clients are expected to interact with it.)
And now, it works. So this is what I can do, in my client code:
LoopbackProcessorHolder.init();
LoopbackProcessorHolder.set(new JettyLoopbackProcessor(localConnector));
Once this is set up, I can use a specific type of URL to connect to the the LocalConnector. If the client connects using that type of URL, it will get a subclass of HttpURLConnection, so it is capable of treating it as an ordinary HttpURLConnection. But in reality, it is not the platform provided HttpURLConnection. It's my own that - using HttpTester - calls into the LocalConnector.
So, this is what the client code would be like, if you happened to be using Hessian:
String url = "loopback:///remoting/HelloService";
LoopbackProcessorHolder.set(new JettyLoopbackProcessor(localConnector));
HessianProxyFactory factory = new HessianProxyFactory();
HelloService service = (HelloService) factory.create(
HelloService.class, url);
System.out.println(service.sayHello());
As you can see, instead of using http://localhost:8080/remoting/HelloService, I am using loopback:/// as the protocol specifier. No host name, no port number, since all of that is not required. The Host header will however be set to localhost, just to be in line with the HTTP/1.1 spec.
A word on Hessian
As I said before, all of this was done to make sure I could test my message authentication scheme, and see how well it would work for authentication of a Hessian client. One of the things I found out that implementing this message authentication scheme into the existing Hessian client code was quite hard. The APIs don't have any hooks to append your own headers. It pretty much has to come from the existing abstractions, or subclasses of these abstractions. And subclassing Hessian classes is harder than it might seem. On the surface, everything is fine. However, there are many cases in which it is actually quite hard to get something sensible done.
Just as an example. HessianProxy is the class that eventually performs the request. Now, ideally, you would simply subclass this class and override a factory method for creating a HessianProxy with your own implementation. Unfortunately, HessianProxy's constructor is package private. So, first of all your HessianProxy subclass always ends up in the same package as HessianProxy itself - which is awkward.
Second, you cannot just override a factory method, since the factory method for a HessianProxy subclass is actually not defined on HessianProxyFactory - which is interesting, given its name. Instead, you have to override a much more general operation called create(), which is expected to return the actual proxying POJO, instead of the HessianProxy. The HessianProxy subclass is created somewhere deep down inside.
I would vote for having a mechanism that is a little bit easier to extend. Out of the box, Hessian only supports basic authentication, which - I think - clearly is not going to cut it. It's much more likely that clients will need some sort of API key that is used to digitally sign messages, rather than sending a password with every request, or obtaining a token once and then continuously send that token back and forth.
