Creating A Custom Liferay Tarball

Okay, so I'm like working on a bigger project that will be a bigger blog sometime soon, but I really need a current Liferay DXP tarball.

When I say current, I mean that it must have:

  • The latest version of Tomcat 9
  • Fixpacks (and possibly hotfixes) applied.

As a DXP client, this just won't be available to you for download.

Even if you are just using Liferay CE, you're stuck w/ the older version of tomcat contained in the bundle.

Sure you can sometimes nab the latest Service Pack release that is also the latest fixpack (until, of course, the next fixpack comes out and then the Service Pack is behind, too). And Liferay ships Tomcat 9.0.6, not something newer...

I didn't want to have to manually do this for my bigger project, I wanted to automate this as much as I can... So I came up with the script I'm going to share, but first let's cover the prerequisites:

  1. You have to manually download a tomcat bundle DXP servicepack. In my example, I've downloaded 7.1 DXP service pack 2. As a client, you can download these from https://help.liferay.com/hc/en-us after authenticating.
  2. Download the fix pack you want to target, also from https://help.liferay.com/hc/en-us after authenticating. Normally you'd go with the latest available. In my example, I've downloaded Fix Pack 12. Also if you have Liferay-delivered support hotfixes, grab those too.
  3. Download the latest version of the patching tool. Currently 2.0.12 is the latest version, you'll find it referenced from https://help.liferay.com/hc/en-us/articles/360016684971.
  4. Edit the script coming up to reference all of these necessary values and versions.

Pick your "source" directory and create a file structure similar to the image below:

Basically it contains a single folder named liferay, and inside of that folder is where you put your Liferay-downloaded DXP tomcat bundle zip file, plus there's also a patches folder and inside there you drop all of your fixpack and hotfix files. If you want to use a newer version of the patching tool, well you just drop that also in the liferay folder.

That's pretty much all you need for the script. It will create another folder, tomcat, where it drops the tomcat tarball you need, and the script will also create a tarball folder where it drops the newly created Liferay tarballs, but both of these things will be handled by the script.

Without further ado, here's the script±:±

#!/bin/bash

#
# This shell script is used to build an up-to-date Liferay DXP Tomcat Bundle tarball with
# fixpacks and hotfixes applied.
#
# This tarball will become the foundation for something more later on...
#
# Note that this script really works with 7.1, it may take some tweaking to work for 7.0 or 7.2,
# but the idea should be pretty straight-forward.
#
# @author dnebing@gmail.com
#

# Where do we normally keep the artifacts used to construct the tarball?
SRC_FILES_DIR=/some/location
# Where do we want to temporarily explode files?
BUNDLE_WORK_DIR=${SRC_FILES_DIR}/work

# What is the version of tomcat we want to use?
TOMCAT_VERSION="9.0.22"
# What is the apache mirror we prefer?
APACHE_MIRROR="http://apache.mirrors.pair.com"

# What is the name of the Liferay DXP bundle zip we will be using?
LIFERAY_ZIP="liferay-dxp-tomcat-7.1.10.2-sp2-20190422172027516.zip"
# When that zip explodes, what is the folder name it will have?
LIFERAY_BUNDLE="liferay-dxp-7.1.10.2-sp2"

# What is the version of the patching tool we are using?
PATCHING_TOOL_VERSION="2.0.12"

# What version label will we give our tarball?
LIFERAY_TARBALL_VERSION="7.1-fp12"

mkdir -p ${SRC_FILES_DIR}/liferay/patches ${SRC_FILES_DIR}/tomcat

# Before doing anything else, make sure we have the files we need...
if [ ! -f ${SRC_FILES_DIR}/liferay/${LIFERAY_ZIP} ]; then
  echo "Liferay Source Tarball is Missing, unable to continue."
  exit 1
fi

if [ ! -f ${SRC_FILES_DIR}/liferay/patching-tool-${PATCHING_TOOL_VERSION}.zip ]; then
  echo "Liferay Patching Tool Zip is Missing, unable to continue."
  exit 1
fi

# If we have not previously downloaded the tomcat tarball
if [ ! -f ${SRC_FILES_DIR}/tomcat/apache-tomcat-${TOMCAT_VERSION}.tar.gz ]; then
  # Download the tomcat tarball now
  wget -P ${SRC_FILES_DIR}/tomcat/ ${APACHE_MIRROR}/tomcat/tomcat-9/v${TOMCAT_VERSION}/bin/apache-tomcat-${TOMCAT_VERSION}.tar.gz
fi

# go to our bundling work dir
mkdir -p ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}
cd ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}

# Explode the tomcat tarball here
tar xzf ${SRC_FILES_DIR}/tomcat/apache-tomcat-${TOMCAT_VERSION}.tar.gz
# And rename the directory
mv apache-tomcat-${TOMCAT_VERSION} tomcat

# Okay, we know there are things in there that we absolutely want to get rid of...
cd tomcat
/bin/rm -f bin/*.bat *.txt *.md LICENSE NOTICE RELEASE-NOTES
/bin/rm -rf webapps/ROOT webapps/docs webapps/examples webapps/host-manager webapps/manager

# Tomcat is now clean, but we need to add some things in...
mkdir -p conf/Catalina/localhost lib/ext/global
touch lib/ext/global/.safeToDelete

# This bundle is now ready for the Liferay overlay...
cd ${BUNDLE_WORK_DIR}

# Liferay tarball should be in the SRC_FILES_DIR...
unzip ${SRC_FILES_DIR}/liferay/${LIFERAY_ZIP}
# At this point Liferay is always using 9.0.6.
cd ${LIFERAY_BUNDLE}/tomcat-9.0.6

# Overwrite files in the target bundle...
# bin...
cp bin/setenv.sh ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/bin
# conf...
cp conf/Catalina/localhost/ROOT.xml ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/conf/Catalina/localhost
cp conf/catalina.policy ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/conf
cp conf/catalina.properties ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/conf
cp conf/server.xml ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/conf
cp conf/web.xml ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/conf

echo "" >> ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/conf/logging.properties
echo "org.apache.catalina.startup.ClassLoaderFactory.level=SEVERE" >> ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/conf/logging.properties
echo "" >> ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/conf/logging.properties

# lib/ext...
cp lib/ext/* ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/lib/ext

# webapps
cp -R webapps/ROOT ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/webapps

# work
cp -R work/Catalina ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}/tomcat/work

# Tomcat is finished, we can move the rest...
cd ..
cp -R data ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}
cp -R osgi ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}
cp -R work ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}

# Liferay is now finished, let's take care of the patching...
cd ${BUNDLE_WORK_DIR}/liferay-dxp-${LIFERAY_TARBALL_VERSION}
unzip ${SRC_FILES_DIR}/liferay/patching-tool-${PATCHING_TOOL_VERSION}.zip
cd patching-tool
cp ${SRC_FILES_DIR}/liferay/patches/* patches

# we need to handle the property file...
cat >default.properties <<EOL
patching.mode=binary
war.path=../tomcat/webapps/ROOT/
global.lib.path=../tomcat/lib/ext/
liferay.home=../
EOL

# Property file is ready, now we can run the patching tool...
chmod +x ./patching-tool.sh
./patching-tool.sh install

# Keep a copy of the patching tool info in case we have to open a support ticket
./patching-tool.sh info > ../patching-tool-info.txt
cd ..

# Patches have been applied, but has now left some cruft behind...
/bin/rm -rf patching-tool
/bin/rm -f tomcat/webapps/ROOT/WEB-INF/patching-backup-deps.zip
/bin/rm -f tomcat/webapps/ROOT/WEB-INF/patching-backup.zip

# Okay, so we should now be all ready to go...
mkdir -p ${SRC_FILES_DIR}/tarballs
cd ${BUNDLE_WORK_DIR}
tar czf ${SRC_FILES_DIR}/tarballs/liferay-dxp-${LIFERAY_TARBALL_VERSION}.tar.gz liferay-dxp-${LIFERAY_TARBALL_VERSION}

# All done with the work dir, let's get rid of it...
/bin/rm -rf ${BUNDLE_WORK_DIR}

I know there's a lot here, but basically:

  1. We create a new directory where the bundle will be built.
  2. We explode the desired version of tomcat into this directory, purging stuff we don't want to carry forward like the default webapps.
  3. We explode the given service pack bundle.
  4. We copy necessary tomcat files the Liferay bundle to the new bundle's tomcat.
  5. We copy necessary Liferay bundle files to the new bundle.
  6. We explode the patching tool zip in the new bundle, copy in the fix packs (and/or hot fixes), configure the patching tool and then apply them to the new bundle. After patching is done, we whack the patching tool as well as the big, unnecessary artifacts left behind.
  7. We create a new tarball from this new bundle, all set and ready to go.

If you're following this to build a current CE bundle, the script can be slimmed down a bit to remove the patch handling, but the rest of the content is still applicable.

The new updated tarball that we just created can be really, really useful.

For example, if you have a Liferay Workspace, you can point the liferay.workspace.bundle.url property in the gradle.properties file to your new bundle (just use the file:// URL syntax) and your local workspace should now use your updated bundle.

If you're building out target non-prod or production environments, this new tarball could be used to set up your Liferay/Tomcat services.

And for my coming blog posts, we'll be relying on this method of creating an updated tarball to do even more cool things...