Blogs

Quite some time ago I was working with a client that had built web service clients as OSGi modules and leveraged JAXB.

During load testing, we found that after many hours of heavy use, the Liferay nodes would stop serving traffic and, for all intents and purposes, would die.  Threads would get hung, blocked waiting to load a class from the web service client, and eventually most of the threads would enter a deadlock state and the node just couldn't recover on its own. The only option we had was to restart the node.

We eventually found and solved the problem and I tucked the details away in case I ran into the issue again. I didn't blog about it at the time because, well, it was the first and only time I had encountered the issue.

Recently though a coworker had run into a similar problem. He too was using JAXB via CXF and was getting blocked threads waiting for the class loader to load a class from the web service client. In his case he was using CXF in a Spring war, but since these wars are converted into OSGi bundles, it was still going to be a problem.

I dredged up and shared the info with him. He did some thorough testing and added an additional property that also helped.

Long story short, if you are doing JAXB web service client code in your OSGi modules, you need to add the following JVM parameters to your app server command line:

  -Dcom.sun.xml.bind.v2.runtime.JAXBContextImpl.fastBoot=true 
  -Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true

That's pretty much it.

If you are currently using JAXB but are not noticing any issues, I think you should still add these properties for the JVM. When I encountered this issue, we were running load tests simulating a heavy concurrent load; if you haven't encountered it yet yourself, it is likely because you haven't yet hit the threshold where things start to fall apart. You wouldn't want to wait until you're under that kind of load to hit this problem as that's the worst time to try to deal with stability issues.  I'd recommend adding the properties right away and just avoid the issue altogether.