Tomcat Session Replication with Redis in 7.4

If you have taken a quick look to this post and believe that it's going to be a nightmare reading the entire text, good news! You can directly jump to Configuration.

 

Things have changed in 7.4 since the previous post was written. While all the explanation regarding Redis is still valid, classloader configuration is a bit different in Liferay 7.4. Anyway we'll perform a similar introduction. This way if someone directly lands in his blog he/she can find everything self-contained.

 

Redis and Redisson

Redis is an in-memory store that can be used to provide a central and external place to save the application session information. It can be very useful in cloud environments or to handle node crashes without losing session information.

Redisson appears as an alternative in order to provide integration between Redis and Tomcat:

  • Provides a Tomcat Session manager that Stores session of Apache Tomcat in Redis and allows to distribute requests across a cluster of Tomcat servers.
  • Implements non-sticky session management backed by Redis.

It has some advantages like writing only the modified attribute instead of serializing the whole session each time. You have more information in Redisson Tomcat.

 

Default Installation

The following steps are needed for a basic installation:

  1. Add RedissonSessionManager: It can be added both on $TOMCAT_BASE/conf/context.xml or, per context, in $TOMCAT_BASE/conf/server.xml. You have more information in Redisson Tomcat.
    <Manager className="org.redisson.tomcat.RedissonSessionManager"
      configPath="${catalina.base}/redisson.conf" 
      readMode="REDIS" updateMode="DEFAULT" broadcastSessionEvents="false"/>

     

  2. Create, in ${catalina.base}, a redisson.conf with a default configuration:
    {
    "singleServerConfig":{
      "address": "redis://127.0.0.1:6379"
    },
    "threads":0,
    "nettyThreads":0,
    "transportMode":"NIO"
    }

     

  3. Copy, in $TOMCAT_BASE/lib, the libraries redisson-all and redisson-tomcat-9 that can be dowloaded from Redisson Tomcat.

 

Why does default installation fails

When starting the portal and perform a log in with any user you'll get a ClassNotFoundException like the following one:

Caused by: java.io.IOException: java.lang .ClassNotFoundException: com.liferay.portlet.PortalPreferencesImpl cannot be found by com.liferay.product.navigation.user.personal.bar.web_6.0.13
    at org.redisson.codec.MarshallingCodec.lambda$new$0(MarshallingCodec.java:148) ~[redisson-all-3.18.0.jar:3.18.0]
    at org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:383) ~[redisson-all-3.18.0.jar:3.18.0]
    at org.redisson.client.handler.CommandDecoder.decodeCommand(CommandDecoder.java:198) ~[redisson-all-3.18.0.jar:3.18.0]
    at org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:137) ~[redisson-all-3.18.0.jar:3.18.0]
    at org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:113) ~[redisson-all-3.18.0.jar:3.18.0]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:519) ~[redisson-all-3.18.0.jar:3.18.0]
    at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:366) ~[redisson-all-3.18.0.jar:3.18.0]

Because Liferay usually programs to an interface not to the implementation the attribute usually will be retrieved and assigned to an interface. So, from the module's point of view there is no need to know how to "reach" the implementation object classloader.

This will cause that the generated MANIFEST.MF for the module (in this case com.liferay.product.navigation.user.personal.bar.web) will only contain Import-Package directives pointing to the interface package, not to the implementation. This is:

  • Import-Package: com.liferay.portal.kernel.portlet (knows how to load the interface PortalPreferences, but doesn't know how to load com.liferay.portlet that contains the implementation PortalPreferencesImpl)

This will be resolved by LiferayRedissonSessionManagerHelper (will talk about it later) saving the context name of the object implementation module classloader during the serialization process. Afterwards, during the deserialization, the classloader can be retrieved from the ClassLoaderPool using that saved context name.

 

Liferay 7.4 Background

In previous versions files like portal-kernel.jar  were deployed in $TOMCAT_HOME/lib/ext folder. This made all these libraries available to each Tomcat deployed application, or even to Tomcat configuration itself, since they were part of the Tomcat classloader. This has completely changed in 7.4 with the aim of having an even more OSGi oriented architecture. More information can be found in Liferay Classloader Hierarchy.

Now the shielded container takes care of making these libraries available to the different modules, but not to outside applications.

Shielded container related stuff can be found in a couple of directories:

  • $TOMCAT_HOME/webapps/ROOT/WEB-INF/shielded-container-lib/: This is just the collection of jars that were available before in $TOMCAT_HOME/lib/ext. These jars aren't directly available to the Tomcat classloader, so they cannot be shared between different applications and they aren't neither available as a single unit. You're just about to see why.
  • $TOMCAT_HOME/webapps/ROOT/WEB-INF/lib: Contains the shielded container files, including ShieldedContainerServletContainerInitializer. This class is responsible for reading all the shielded-container/lib jars and making them available through the ShieldedContainerClassLoader, so that the previous jars classes are part of a single classloader, the ShieldedContainerClassLoader.

 

This would normally not affect your own developments or your infrastructure configuration. At much you should consider making the database drivers available globally moving them to the Tomcat lib, just in case you need to share them between applications.

So, at this point, you only need to remember that Liferay non OSGi application jars are no longer deployed in $TOMCAT_HOME/lib/ext.  That makes them not shared globally. The only Liferay classes that DXP web application interacts with are Liferay’s Shielded Container JAR files: com.liferay.shielded.container.api.jar, com.liferay.shielded.container.impl.jar

 

What's different in this solution?

Previous solution was based in LiferayRedissonSessionManager. Adding it as a manager in context.xml or server.xml files made it possible to handle serialization/deserialization between Liferay and Tomcat.

Unfortunately Liferay classes needed by LiferayRedissonSessionManager aren't visible anymore from the Tomcat lib classloader. And because the LiferayRedissonSessionManager is configured through Tomcat configuration Liferay classes used by the manager aren't accessible.

 

Two way based approach

So, somehow, the new idea is oriented towards the need to coordinate Tomcat and Liferay communication decoupling, as much as possible, Liferay classloader code from Tomcat Redisson integration code.

Tomcat side classes

  • LiferayDelegatedRedissonSessionManager:  Extends RedissonSessionManager with the purpose of delegating on Liferay the serialization/deserialization process. This is done through composition, making the API able to receive a the LiferayRedissonSessionManagerHelper that will perform the Liferay's side serialization/deserialization. Until this helper is received the code will wait for it using a CountDownLatch
  • RedissonSessionManagerHelper: Helper that will be implemented by LiferayRedissonSessionManagerHelper. Just contains two methods, one for creating an empty session and one that will return the codec that will contain an additional Liferay classloader aggregated.

 

Liferay side classes

The idea is to let Redisson handle how to save/retrieve data from Redis letting Liferay manage the serialization/deserialization process since Liferay knows how to reach each module appropriate classloader. 

  • LiferayFstCodec: Hasn't changed at all from previous version. Just adds to the current Redisson classloader (application context, thread or class classloader) Liferay's portal default classloader, PortalClassLoaderUtil#getClassLoader(). This makes access to some of Liferay core classes possible.
  • LiferayRedissonSessionManagerHelper: It's the real bridge between Liferay and Redisson. Uses LiferayFstCodec to have a gate to Liferay classloaders and manages serialization/deserialization using Liferay Serializer/Deserializer. Liferay serialization handlers are based on the ClassLoaderPool that contains each module's classloader, which know how to handle their classes serialization/deserialization. This class is based on the former (used up to 7.3) Liferay PortletSessionImpl with a slight improvement for Bootstrap loaded classes (Byte.class, String.class, Class.class,...)
  • RedissonShieldedContainerInitializer: Biggest change from previous version. Initializes Tomcat session replication class with LiferayRedissonSessionManagerHelper. This is done as soon as the ShieldedContainer gets initialized, meaning that its classloader is available (all former portal-kernel, portal-impl are reachable through this classloader). From the moment Tomcat starts until the ShieldedContainer is initialized Tomcat replication code will wait using a CountDownLatch.

 

Configuration

All code can be downloaded from Liferay Session Replication with Redis in 7.4.

Next steps are needed to make the integration work:

 

  1. Change the configuration to use custom Liferay manager in $TOMCAT_HOME/conf/Catalina/localhost/ROOT.xml:
    <Manager className="com.liferay.redis.redisson.integration.tomcat.LiferayDelegatedRedissonSessionManager"
      configPath="${catalina.base}/redisson.conf"
      readMode="REDIS" updateMode="DEFAULT" broadcastSessionEvents="false" broadcastSessionUpdates="false"/>
    
  2. Configure Redisson with Redis address in $CATALINA_BASE/redisson.conf:
    {
    "singleServerConfig":{
      "address": "redis://127.0.0.1:6379"
    },
    "threads":0,
    "nettyThreads":0,
    "transportMode":"NIO"
    }
  3. Deploy redis-redisson-integration module to $TOMCAT_HOME/webapps/ROOT/WEB-INF/shielded-container-lib
  4. Deploy redis-redisson-integration-tomcat to $TOMCAT_HOME/lib

 

Logging

If you want to avoid annoying Redisson INFO traces like the following ones:

28-Nov-2022 15:49:04.702 INFO [http-nio-8080-exec-6] org.redisson.tomcat.RedissonSessionManager.findSession Session 8040B36EC63A98DCDC21865401E1FF1E can't be found
28-Nov-2022 15:49:39.180 INFO [http-nio-8080-exec-2] org.redisson.tomcat.RedissonSessionManager.findSession Session B0D46572F74B0BD4CE9DE0E9552BD0CD can't be found
28-Nov-2022 15:49:39.196 INFO [http-nio-8080-exec-2] org.redisson.tomcat.RedissonSessionManager.findSession Session B0D46572F74B0BD4CE9DE0E9552BD0CD can't be found
28-Nov-2022 15:49:39.214 INFO [http-nio-8080-exec-2] org.redisson.tomcat.RedissonSessionManager.findSession Session B0D46572F74B0BD4CE9DE0E9552BD0CD can't be found
28-Nov-2022 15:49:39.299 INFO [http-nio-8080-exec-9] org.redisson.tomcat.RedissonSessionManager.findSession Session B0D46572F74B0BD4CE9DE0E9552BD0CD can't be found
28-Nov-2022 15:49:39.299 INFO [http-nio-8080-exec-9] org.redisson.tomcat.RedissonSessionManager.findSession Session B0D46572F74B0BD4CE9DE0E9552BD0CD can't be found

You can configure your Tomcat logging to display only WARNING level traces in $TOMCAT_HOME/conf/logging.properties

org.redisson.tomcat.level=WARNING

 

 

This was done mostly on my free time. Don't be afraid, I'm not asking for money back in case you decide to use it.  But it would be nice if your organization spends some of the development money saved in a non-governmental organization that can help to make the world a better place.
Special thanks to Juan Carlos Hernández Ramos who tested the final solution on his company environments.