RE: Replacing SSL certificates without restarting/downtime in Liferay DXP

Santanu Kumar Giri, modified 6 Years ago. Junior Member Posts: 34 Join Date: 6/6/11 Recent Posts
Hi,As we have requirement to change the certificate for liferay DXP(Tomacat) without restarting the tomcat.We tried to restart tomcat connector programmatically and it looks fine , But after restarting the Connector we are not able to login to the portal.As per my understanding the connection and session has lost  . After restarting the tomcat server manually all looks good.
But our requirement is no server restart . Please suggest me how to resolve the certificate reload issue. Thank you
thumbnail
Christoph Rabel, modified 6 Years ago. Liferay Legend Posts: 1555 Join Date: 9/24/09 Recent Posts
How often do you really plan tho change the certificate? If you plan to do it on a regular basis, e.g. when using certbot, I really recommend to setup a reverse proxy and put it in front of the tomcat. As a rule of thumb: You probably should have a reverse proxy anyway in front of your tomcat because it provides security benefits. A reverse proxy can withstand DDOS attacks far better. (I hope, that's still true emoticon )
It also has other benefits, among them easier and faster SSL configuration. Changing a certificate on the fly is rather trivial in nginx/apache or any other reverse proxy servers.
Santanu Kumar Giri, modified 6 Years ago. Junior Member Posts: 34 Join Date: 6/6/11 Recent Posts
Thanks for the quick reply !!Currently we have not used in reverse proxy. We have created a docker image for Liferay DXP which have been hosted in Kubernetes platform. We have one liferay pod which is serving all client request. Could you please give me some idea how to do the same in Kubernetes without restarting the pods.
thumbnail
Andrew Jardine, modified 6 Years ago. Liferay Legend Posts: 2416 Join Date: 12/22/10 Recent Posts
I'm no expert in Kubernetes but wouldn't it just be the same thing as with out? Either you add the proxy to your Liferay docker image, or better yet, you create a separate image that contains just your nginx/apache reverse proxy and wire it into your env such that traffic flows to your Liferay configuration through your  nginx/apache pod. In your current setup, you have no proxy at all?
Santanu Kumar Giri, modified 6 Years ago. Junior Member Posts: 34 Join Date: 6/6/11 Recent Posts
Currently we don't have any reverse proxy in front of our tomcat server  that's why I think server restart is the only option to fetch the updated SSL certificate.Our site is not yet in Production we just enabled the SSL and Customer has asked input on restoring the SSL certificates . I was going through the possibility of  restoring the SSL certificates. Kindly suggest me if some other way is there. Thank you
thumbnail
Christoph Rabel, modified 6 Years ago. Liferay Legend Posts: 1555 Join Date: 9/24/09 Recent Posts
I have no idea if the tomcat connector can be restarted. Maybe it's possible, but I wouldn't go there. It's very uncommon and it could cause weird problems. In any case, you are probably better off if you ask that question in a dedicated tomcat forum.
I can only recommend again to add a reverse proxy to your setup. It might seem hard now, but believe me, a reverse proxy is fantastic.
Santanu Kumar Giri, modified 6 Years ago. Junior Member Posts: 34 Join Date: 6/6/11 Recent Posts
Thank you !!  Christoph  and Andrew 
Santanu Kumar Giri, modified 6 Years ago. Junior Member Posts: 34 Join Date: 6/6/11 Recent Posts
Hi,

I discovered now that We already have a reverse proxy (Ingress Controller) which does TLS termination for all public APIs exposed by LR. So, that setup is already there.

Where we are facing problem is internal machine-to-machine communication with Liferay, which is also TLS based and there we don’t need to go via a reverse proxy since that’s direct internal M2M communication between microservices.
thumbnail
Christoph Rabel, modified 6 Years ago. Liferay Legend Posts: 1555 Join Date: 9/24/09 Recent Posts
Ok, in that case:

Why do you need to change the certificate so often? Why don't you set the expiration time to two years or something else you think is manageable? You also can create your own internal CA (In a large company you probably already have one) and sign your tomcat certificates with that certificate. Than it has not cost and with a little effort still works.
Santanu Kumar Giri, modified 6 Years ago. Junior Member Posts: 34 Join Date: 6/6/11 Recent Posts
Our certificate management for M2M communication is automated which has a CA that generates internal certificates, and renews upon expiry, for maximum duration of 1 week for better security.

Hence it's essential for us to be able to automatically reload certificates upon expiry with minimum downtime. 
thumbnail
Andrew Jardine, modified 6 Years ago. Liferay Legend Posts: 2416 Join Date: 12/22/10 Recent Posts
I'm just curious about something so I thought I would ask. Restarts happen right. For me and my clients, we've just always established a practice of controlled restarts. So let's say I have a 3 node cluster. Take down one node, make the changes, bring it ack up .. then take down the other two nodes and make the same updates and bring them back up. That way there is always at least one node online to server requests. I guess I have been fortunate enough in my past to have systems where a single node could handle the traffic (off peak hours of course) and I've discouraged the use of "session storage" so that users can bounce between nodes without weird stuff happening. To date, this has been a reasonable solution -- do you think something like this might work for you?
thumbnail
Christoph Rabel, modified 6 Years ago. Liferay Legend Posts: 1555 Join Date: 9/24/09 Recent Posts
Puh.
1 week is really brutal. Not sure, why this should in any way increase your security, but if your company has that policy ...

I see the following options:
- Set up a reverse proxy on your Liferay servers and install the certificate there
- You discuss the requirements with your security team. I have never seen 1 week anywhere before.
- You live with the downtime
- Maybe Andrew's advice helps and you can restart them in turn
- Maybe a different application server (Glassfish?) allows you to do what you want and you switch to it
- You actually solve that "recreate listener" issue

I still believe, the first solution is the easiest to implement.
thumbnail
Olaf Kock, modified 6 Years ago. Liferay Legend Posts: 6441 Join Date: 9/23/08 Recent Posts
I've not seen 1 week expiration of certificates. And I don't see how that is "for security". Especially on an internal CA, you'd be able to do revocation checks without the internet-scale checks bringing your CA down. If, through a security check, you realize that your key has been compromised on day 1 of the valid 7 days, you'd want to revoke it anyways, so you'll need the revocation process anyway.

Anyway, the whole configuration would be on your appserver (tomcat?) side anyway, not on Liferay's side, as Liferay doesn't handle any request by itself, but gets them passed in from the appserver.

You may also want to read about OCSP, and rather configure it with a short lifetime, which is a nice alternative to revocation checks. That, as well, is to be configured on your appserver, not on Liferay.

I'm assuming that the CA's root signature is valid for several years, as is common, and only the individual certs are short lived.

If you can't go with any of the strategies on the appserver: Set up a Liferay cluster: This way one machine can keep up the traffic while the other is restarted. In the restart process, implement a cert renewal. When you have a cluster, you'll need a reverse proxy (load balancer), and then you can terminate TLS there - so you can omit the cluster again and continue running on one machine, just add a reverse proxy. And we're back at the beginning ;)