Blogs
My friend, Olaf Kock, recently shared with me that he had struggled with and resolved an issue after moving to Tomcat 9.0.31 when using AJP.
Update: What maybe we didn't know, Tomcat 9.0.31 (and other versions of Tomcat 6, 7, 8 and 8.5) were all being fixed to address a newly identified attack vector against Tomcat nicknamed Ghostcat: https://www.zdnet.com/article/ghostcat-bug-impacts-all-apache-tomcat-versions-released-in-the-last-13-years/
Since I'm a big fan of using AJP to connect Apache HTTPd and Tomcat, I thought I'd share what he found with you. I must say, though, that this is just a blog about Olaf's efforts, I'm now just playing Watson to his Holmes...
Long story short, in Tomcat 9.0.31 (and onward), the AJP connector is not going to be enabled by default. It was removed to prevent exposure as a security attack vector. There were also some changes to configuring AJP correctly.
I'll present these shortly, but wanted to summarize first...
Why AJP?
I think typically you'll find most often that sys admins front tomcat with Apache HTTPd using mod_proxy, basically a mechanism where httpd accepts the incoming HTTP connection on port 80 and will proxy an HTTP request to tomcat running typically on port 8080. Port 8080 is open and the default Tomcat port and it's why we all just spin up a Tomcat instance and will use 8080 in our browser URLs for testing.
Because this is just an OOTB configuration for both HTTPd and Tomcat, it is really an easy way to connect the two. Oh, and Apache doesn't normally ship the AJP connector with HTTPd, so there's extra effort to get AJP connection going in your environment.
So why does it make sense to try and pursue the AJP option?
First, it's a connection leveraging a binary protocol. The headers, cookies, parameters and other values in an HttpServletRequest? Those get parsed by Apache HTTPd already. Using AJP, the pre-parsed form is passed on to Tomcat for processing, unlike with the proxy method where Apache HTTPd is parsing, but then Tomcat also has to do the parsing again.
Second, Tomcat sees the request as though it came straight through to Tomcat, without Apache HTTPd in the middle. So the source address, port, and other information are given to Tomcat as-is. With the Proxy option, the request that Tomcat gets originates from the HTTPd server, not the remote client. So you have to go through additional configuration to expose things like the incoming URL, the source IP address and port, etc. to Tomcat.
To me, these two things alone make AJP a clear win, even though it has the extra deployment and setup concerns.
What Changed?
Rather than rewording myself, I'm just going to quote Olaf since he did all of the work:
As soon as our bundles are updated to Tomcat 9.0.31, note that there were some changes in the AJP Connectors that might disrupt people's expectation: They're disabled by default, and enabling them requires setting an interface explicitly, as well as a secret that must be shared with the reverse proxy.
Those who don't use AJP don't need to do anything, those that use AJP should notice this "secure by default" configuration and make sure that they don't open a security hole by re-enabling. Relevant sections from https://tomcat.apache.org/tomcat-9.0-doc/changelog.html
- Disable (comment out in server.xml) the AJP/1.3 connector by default. (markt)
- Change the default bind address for the AJP/1.3 connector to be the loopback address. (markt)
- Rename the requiredSecret attribute of the AJP/1.3 Connector to secret and add a new attribute secretRequired that defaults to true. When secretRequired is true the AJP/1.3 Connector will not start unless the secret attribute is configured to a non-null, non-zero length String. (markt)
- Add a new attribute, allowedRequestAttributesPattern to the AJP/1.3 Connector. Requests with unrecognised attributes will be blocked with a 403. (markt)
Caused a few confusions that were discussed on the tomcat mailing list.
So basically to continue to use AJP, you're going to be editing the default server.xml that Tomcat ships with to enable the connector, possibly bind to a network address (if HTTPd is on another server), plus include these attribute changes highlighted above.
So, two important points here:
1. If you update your Tomcat server to 9.0.31 (or later), you are going to need to make some changes to the configuration for these new AJP updates.
2. If you see Olaf out and about, you should thank him (as I do) for gathering this information for us...
Update
Since posting the blog, news of Ghostcat has been spreading: https://www.zdnet.com/article/ghostcat-bug-impacts-all-apache-tomcat-versions-released-in-the-last-13-years/
Every system admin, whether using AJP or not, needs to make changes in their environment:
- If not using AJP, disable the AJP connection on port 8009 in server.xml. It's also a good idea to block external access to the port in case the AJP gets re-enabled accidentally in the future.
- If using AJP, update your Tomcat version to the newly updated ones with the patched AJP code.
- Establish firewall rules that only allow AJP connections from trusted hosts.