So recently I completed a database upgrade from Liferay EE 6.1 up to Liferay DXP 7.4. It was actually pretty straightforward and I didn't encounter any significant blockers.
I did however learn some stuff along the way and I thought I'd share those details here so you will be able to do your next Liferay DB upgrade in the same way.
First and foremost, you must have a plan. You shouldn't walk into an upgrade cold thinking you're just going to do it like it is no big deal.
My plan that I'm going to share with you has 7 specific steps. At the end of each step I had completed an important task and I had the artifacts to show I was ready to move on.
Your plan might not be as regimented and detailed as mine, and that's fine. It's important that whatever your plan is, you've thought about it and how you're going to get through it from start to end.
I always recommend to clients that they get a copy of their production database and Docs and Media (D&M) folder and have the upgrade team run a few upgrades against the environment.
This helps the team in the following ways:
The plan I'm presenting also includes a number of backups and possible restores. This can seem like a pain, but ultimately these will save you time and effort.
For example, to upgrade from 6.1 we actually have to do two upgrades, one from 6.1 -> 7.1, and the second from 7.1 -> 7.4. After the 7.1 upgrade is complete, my plan grabs a backup of the database and D&M in case there is an error in the 7.1 -> 7.4 upgrade that forces me to roll back and apply a SQL change. I want to be able to roll back to the 7.1 version and go from there. Without it, I'd have to roll all the way back to 6.1, then I'd have to complete the 6.1 -> 7.1 upgrade again, all before I could fix my data and restart the 7.1 -> 7.4 step.
Obviously I won't know there is a data issue that might break the 7.1 -> 7.4 upgrade, but if it happens I am prepared because I have planned ahead.
One important note about backups - the restore may not be enough on its own.
The 6.1 -> 7.1 upgrade creates a new table, LayoutFriendlyURL and it populates the table from values from the Layout table. In some databases and for some types of backups, if I were to restore the database the Layout table can be reset to what it was, but the LayoutFriendlyURL table might not be removed and/or purged, and this can actually cause problems when the upgrade runs again.
So when I do a restore, I do a clean restore; I drop all of the tables in the database and I delete the D&M folder, then I will restore the DB and the D&M from the backup. This ensures that I won't have any lingering artifacts from a previous failed upgrade.
So I needed to create my plan and I knew that I was starting from Liferay EE 6.1 and ending at Liferay DXP 7.4. I started by checking https://learn.liferay.com/dxp/latest/en/installation-and-upgrades/upgrading-liferay/upgrade-basics.html#review-available-upgrade-paths for my available upgrade paths.
From 6.1 I found I have to complete a two-phase upgrade, phase 1 is 6.1 -> 7.1, and phase 2 is 7.1 -> 7.4.
Using this as the foundation, I was ready to create my plan...
So here's my plan in all of it's glory.
Here my goal is to create a new 6.1 database (potentially restored from the prod database) and use a 6.1 bundle with the D&M (also restored). And I don't want to just create the environment, I want to fire it up and make sure it is able to connect to the database and expose all of the D&M files.
With this ready, I at least know I'm starting the upgrade from an environment that works.
This preparation is also detailed and requires me to complete the following:
Ultimately this step involves getting together everything necessary for performing the next step, like a doctor preparing her tools for a coming surgery or a chef taking out the ingredients for the next meal.
The script itself is in the tools/portal-tools-db-upgrade-client directory and you'll either use the Un*x version or the Windows version depending upon your platform.
When I run the script, I use an extended command line to set JVM parameters to avoid runtime memory issues (and possibly forcing a rollback and restart). The command I use is:
./db_upgrade.sh -j "-Xms8g -Xmx8g -XX:MaxNewSize=1536m -XX:MaxMetaspaceSize=1g \
-XX:MetaspaceSize=1g -XX:NewSize=1536m -XX:SurvivorRatio=7"
I have a pretty beefy system so I can throw 8g at the upgrade process without too much concern.
Although this step, "Run the db_upgrade script", sounds like a simple task, it is really more of an iterative process. Here's the flow that I follow:
I will iterate over these steps until I get to a point where the db_upgrade script doesn't fail and doesn't report errors on data that I don't already know about.
Why do I go through all of this? The most important thing for a working, functioning Liferay system is the database. The data in there represents all of your pages, all of your web contents, all of your documents, your blog posts, message boards, configuration, etc. A failure in the DB upgrade, even a small one, can have a pretty significant impact in the runtime and stability of your upgraded environment.
So I take every failure seriously. I'll research the Liferay code behind an exception and try to figure out why it failed, even getting help from Liferay Support if I need to. Anything to get to a clean db_upgrade script run.
That last run is the source that you use for estimating your upgrade times. It's only an estimate because your local environment is probably a different class of machine from your production environment. Even so, if it takes 12 hours on your local you can be pretty sure that it will take a big chunk of time in production and you can probably also use this as your "worst case scenario" upgrade time for this step.
Yep, it's that time. At this point we have finished the 6.1 -> 7.1 portion of our upgrade, and taking a backup here means that if the next portion of the upgrade fails, I can fall back to this point rather than all the way back to the start.
If the next portion doesn't fail, I've lost nothing but a small bit of time for the backup. But if the next portion does fail, I'll save all of the time from previous upgrade run, and you know by now what that time is going to be.
This is effectively the same set of tasks that we did in Step 2, but this time we're doing the same for the latest 7.4.
The one difference, we need to pull forward the data/document_library folder from the 7.1 environment to the 7.4 environment. Since it has already been upgraded, it is the D&M that needs to be used to continue the upgrade to 7.4.
The properties files should be copied in also and edited for the 7.4 environment.
For 7.4, the JDBC jar should be copied to the tomcat/webapps/ROOT/WEB-INF/shielded-container-lib instead of tomcat/lib/ext.
The process here is going to be the same as the one in Step 3, but we're going to using the 7.4 version of the database upgrade client. We use the same command line (although in the different directory).
Most importantly, we're going to follow the same process: run the script, analyze the results, build a cleanup SQL script, and wash/rinse/repeat until we get a successful run of the upgrade process.
And we do this for the same reason as in Step 3, we want to set a solid foundation for the 7.4 environment by starting from an error-free database upgrade.
May seem strange to take a backup after you're done, but I think it is a great time to get a backup.
You'll be capturing a snapshot of the database and D&M folder after the DB upgrade is completed but before you are starting up your 7.4 environment. Having the backup at this point gives you a fallback point if your 7.4 development or deployment goes sideways.
So the important part here, at least for me, is that I have a plan to complete the upgrade. I've documented the plan so I or any member of the team can follow. It's also repeatable because the steps are clearly documented.
The process itself also focuses on getting a clean database upgrade as the foundation of the 7.4 environment. A clean database upgrade increases the probability of an overall successful upgrade.
It's important to remember though that the successful db upgrade client run does not imply that there might not be issues in 7.4. Further testing with your customizations and your content is still necessary to validate your updated environment is ready.
But at least you'll be able to start this testing and validation process knowing you're starting from a solid foundation.
I hope this plan and process will help you when it comes time to upgrade your own environment to Liferay CE or DXP 7.4.