Friday, August 3, 2012

REST APIs served twice on port 80( via IIS/AJP Connector)


Few months back I faced an interesting issue related to REST APIs. When REST APIs were fired on port 80(via IIS, ajp connector), they were being served twice by the server. However when it was fired on port 8080(hit Tomcat directly via HTTP 1.1 connector), it got invoked just once. I verified using HTTP inspection tool that client was sending a single request to server and so was IIS.

The REST API framework was based on Apache CXF so I posted the question on their forum to check if they had any clue on what was going wrong.
Didn't get much help from that so after all analysis,troubleshooting and reading online forums, this is what it turned out to be-

For each API HTTP request Apache CXF sent back an extra packet of response which unfortunately did not adhere to the IIS connector's (AJP) protocol. This threw an exception in connector code which blocked the response midway.After it recovered from the exception it ignored(no rollback) the previous response and re-fired the same API request again which worked - hence we had two invocation of the server code but client got a single response based on second invocation. This proved hazardous for create APIs as second response was mostly unique constraint errors.

I noticed that this issue was reported as a conenctor bug a while back, as its expected to handle these failures more gracefully - https://issues.apache.org/bugzilla/show_bug.cgi?id=48826. When I picked up the attached dll in this bug it didn't work for me, there were more exceptions thrown.
I was then looking through CXF code to find source of extra packet of response but that didn't turn out to be easy .. also got in touch with CXF team and they asked me to switch to simple HTTP proxy setup as they have seen quite a few issues with AJP!!

However when I pick up the latest dll from here http://apache.mirrors.tds.net//tomcat/tomcat-connectors/jk/binaries/windows, the problem goes away and everything gets normal. Looks like this bug is address in the version 1.2.37

So finally updating the IIS connector resolved the bug for me and saved my APIs from crashing!

No comments:

Post a Comment