Introduction

There are a lot of things that Liferay just won't say.

They won't tell you what app server to use, won't tell you what framework to use, won't tell you how to deal with your production rollout or your production deployments, etc.

There is really a lot of stuff that they won't say, but you don't really get the impression that there are things that aren't said by reviewing the doco. If you spend time in the doco, especially the client doco available to DXP clients, you can step away thinking that you may know all of the unsaid things, but I don't think you really do. After all, they are unsaid.

So I thought I'd take some time out to talk about the things Liferay doesn't say, give my guesses as to why they don't say the things, but then share with you my responses as though we were sitting at the bar sharing a drink or three...

What App Server Should We Use?

This one comes up a lot. Liferay will not say which one to use, and in fact there are bundles available for many of the open source options.

My guess as to why they won't make a choice here, it simply comes down to adoption. If Liferay were to say "Liferay DXP Runs Best On Xxxx", if you are not running Xxxx then you might not want to give Liferay a look. It helps that Liferay runs on all of the common application servers, and most have supported configurations.

My opinion? Use Tomcat. It is cheap. It has all the functionality that Liferay requires. It is easy to install and maintain, heck most flavors of Linux can install Tomcat using the native package manager.

The enterprise JEE servers like WebSphere, WebLogic, and JBoss have little to offer a DXP deployment, especially since all Liferay deployments now have you dropping the artifacts in the Liferay deploy folder and skipping the centralized console altogether (unless you use the Deployment Helper, and I don't know anyone who is).

If you need support (which no one really needs support for Tomcat, it is so prevalent that most issues can be resolved with a Google search or two), then I would point you at TCat, a version of Tomcat that comes with a support option.

What Database Should We Use?

Again, Liferay won't make a suggestion here. Liferay has layers around Hibernate, so it is effectively database neutral. Like the app server question, Liferay won't make a recommendation so they don't dissuade an organization using a different database.

My recommendation has changed over time. I used to recommend MySQL, but then with the whole fork mess I looked at MariaDB, but at this point I'm partial to PostgreSQL. Like Tomcat, it is easy to install and administer, and it too can be deployed via many Linux package managers. It has support for backup/restore, handles replication and has all of the necessary features for hosting Liferay without coming with a hefty pricetag.

What OS Should We Use?

So this one tends to be the opening question before a holy war; everyone has their own favs here.

And like the App Server and the Database questions, Liferay will not make a recommendation here so they don't shut out a sector that they will happily run under.

Here, my only response is "Anything But Windows". Seriously. Anything else. Not Windows.

How Much Memory Should I Use? (AKA How Do I Size My Node(s))

Many people are surprised to find that there just are little in the way of recommendations of any kind for Liferay sizing, and they often ask why.

It's not that Liferay is trying to hide the requirements or anything, and it is not that they are afraid that the sizing requires something significant that would come with sticker shock.

The answer here is always "It Depends"™. Honestly, you will have all of the information about sizing, much more than Liferay does. Sizing depends upon your own criteria.

Sizing depends upon two factors: content and capacity.

For content, sizing depends upon the answers to questions like how many documents you'll have, how many web contents, will you be hosting videos, etc.

For capacity, sizing depends upon the answers to questions like what is your expected user count, what is the expected concurrent user count, what are the required response times, etc.

Liferay will never give sizing recommendations out because, after all, they can't know in advance how you are going to answer these questions. And they can't make statements like "A node will handle 1,000 concurrent users" since custom code, node sizing, integration performance, etc can all impact overall node capacity.

As a general rule of thumb, I typically suggest starting with 8gb (12 or 16gb if not VMs or in cloud) node(s) leveraging "mid-tier" server architecture (larger if you have deep pockets). That seems to handle most use cases nicely, then you can add nodes to deal with capacity concerns, front side caching, etc.

Along with that I would also add a 3-node Elastic cluster (3 at a minimum because of split brain, but if you are a heavy search user I'd go for 3+).

Then I also plan for the surround stuff, Apache HTTPd servers to front the Tomcats, Varnish servers thrown in for front side caching (to reduce load on the app server), ...

Does Liferay Work in The Cloud?

I'm not sure really if this one needs to be here, but as I thought about it I'm not sure I could easily find a reference in the documentation to say one way or the other, so I just decided to answer it.

Yes!

I think Liferay has been kind of slow with the creation of things like Docker containers, etc., that would have made launching in the cloud easier, but that doesn't negate the possibility.

You need the same things you would need for local hosting - EC2 instances for the app servers, EC2 instance for the Elastic servers (no, you can't yet use the Elastic cloud service, it doesn't have some necessary plugins), an S3 bucket for doc store (please permission the bucket correctly, don't go allowing public access to the doc store) plus RDS w/ your DB. You'd want to use JDBCPING for clustering to enable autoscaling.

You'd need the equivalent stuff for Azure or Google clouds, so no real perceptive difference.

It's a lot of manual work, certainly, but some can be avoided by choosing DXPCloud.

What JVM Parameters Should We Use?

This one is akin to the question about sizing.

Liferay provides guidance from the Deployment Checklist. It has a big set of initial settings that, unless you are a java developer, will likely make no sense to you at all.

I can't tell you how many times I have found clients using the verbatim list of parameters and, when I ask them how they configured the large pages in their unix environment, I get blank stares as a response.

That's okay, though, but just a clarification on the checklist parameters. Liferay doesn't mean for those to be your permanent JVM parameters, they are merely a starting point they recommend you start with. In a perfect world, you have your analytics tools and load testing tools ready to go before you launch production, and you use these tools to tweak the JVM knobs to get the most out of the JVM, and the settings you end up running with you can back up from the results of the load testing that you completed.

But, in the non perfect world, the JVM parameters from the checklist are certainly a better list than what you get from a standard bundle.

So my recommendation is to at least use the values from the checklist, plus I recommend doing the load testing to see if you can turn the knobs and get a better response. At the end of the day, the parameters will impact your capacity thresholds so, the better numbers you can get, the higher your capacity is going to be.

Can We Change the JVM Timezone Setting?

It sure may seem like an easy thing to do, and it's not like Liferay doesn't say not to do it...

But I will.

No, do not change the user.timezone property for the JVM.

When you are dealing with traffic from around the world, sticking with UTC (GMT) keeps all date displays super simple; convert UTC to user's timezone and you're done. If the dates are not in UTC, then you have a two-way conversion, server TZ -> UTC and then UTC -> user's TZ. This is because there is no easy table that shows how to convert directly between all of the timezones around the world.

If you are only serving traffic to folks in your current TZ, it can seem like it is not a problem because TZ conversions wouldn't be necessary.

But here's the thing - some internal Liferay stuff actually does kind of depend upon TZ things. Quartz, for example, if you schedule a job at 10am CST, it is not automatically going to run at 11am in CDT because it may not recognize the TZ change.

Historically there have been bugs in Liferay relative to the JVM TZ change. While those have been fixed, I don't believe Liferay actively tests in non-UTC timezones so there may be new or future bugs as a result.

I always recommend leaving the JVM at UTC, just change the company timezone and user timezones to your local environ and the times should display correctly for the users.

What Maintenance or Administration Tasks Don't I Know About?

Running through the Liferay documentation, there's basically an underlying assumption that you have existing knowledge and expertise behind your chosen OS, App Server, Database, etc. With that expertise, you will be able to stand up a working and stable cluster and everything will be puppies and flowers.

But, if you don't happen to have that expert available when you read this, these are the kinds of things which I will recommend:

• Clustering: Use TCPPING in house or JDBCPING in the cloud; don't rely on UDP because multicast doesn't always end well.
• NAS: Get your network storage squared away so all nodes are sharing the same store.
• Backups: Have a backup strategy for the database and the document store. As my friend Olaf Kock says, you don't have a backup unless you test it, so be sure to test your restore process so you verify the effectiveness of your backup.
• Disaster Recovery: Only you know if you need a DR plan. If so, that plan should include a DB replica as well as the document store replica. You won't need a replica of your Elastic, all you need to do is reindex when the DR is kicked in. This too is like the backup, you don't know if you can recover from a disaster unless you fully test your DR plan.
• Updates: You not only have OS-level updates to apply, but also updates to your app server, your database, Elastic, Liferay, etc. These are continuous things that you should consciously decide when you are not going to be doing them, but don't falsely believe that once the node is set up, your work is done. With a strong firewall and strict zoning (i.e. DMZ vs internal), you might be able to get by with the "if it works, don't fix it" mentality. But most of the time I recommend having a plan for how to deal with the updates for all of these pieces. As I've said before, I don't believe the Liferay bundles are suitable for a production deployment specifically because the versions are just too darn old and therefore have some vulnerabilities baked in.
• Log Rotation/Cleanup: Liferay default configuration includes log rotation, but does not do any form of log cleanup. You should have a plan for dealing with log cleanup in production.
• Audit Cleanup: The Liferay audit mechanism dumps records out to the Liferay database in its audit tables. There is no automatic purge in there, so have a plan to deal with that if auditing is enabled.
• 3rd Party Tools: I normally want Ghostscript, ImageMagick, LibreOffice (OpenOffice) and Xuggler installed. This comes with additional installation requirements and ongoing maintenance, but they are worth it when it comes to indexing and converting content and documents.
• Firewall: A secured server is going to batten down the firewall, only opening holes to approved ports/services, and sometimes only from approved sources. So you don't have generic SSH enabled into your app server, but maybe only SSH on a specific port and from an internal system or jump box. This way you manage your exposure and limit internal service vulnerabilities.
• SSO: Even if you don't have an SSO solution currently, it's pretty darn easy to have one. Even if you just configure Liferay to be the IdP, it still gives you an SSO solution that you can use across your organization.
• APM: Application Performance Monitoring is critical to know what is going on in your environment, to prevent outages by seeing what is going on in the environment before things actually happen. I'm kind of partial to DynaTrace, but AppDynamics, Introscope and NewRelic will all give you visibility beyond just simple health checks.
• Health Checks: Even though they are simple, they are still necessary. You want your LB to be able to determine availablility, and a health check is a great avenue to allow that to happen.
• Hardening/Security: Each application server has its own aspects related to hardening and security. These aspects should be integrated into your production environment.  Olaf did a nice blog on hardening Tomcat: https://web.liferay.com/web/olaf.kock/blog/-/blogs/securing-liferay-chapter-1-introduction-basics-and-operating-system-level.

Does Liferay Conform to <Insert Accessibility Guideline Here>?

This tends to come up a lot for organizations that need to comply with accessibility standards.

Liferay doesn't really say that they do or don't conform to any guidelines, so it is really not clear.

In reality, the answer is Yes and No.

So an OOTB Liferay with the classic theme and OOTB portlets and web content will be compliant with most accessibility standards.

Why don't they say so?

Because  nobody uses the OOTB Liferay w/ the classic theme. If you build a custom theme but don't include the link to allow speech-based browsers to skip the <nav /> area, you won't be compliant. If you add a web content article but don't include the necessary attributes on an embedded image tag, you won't be compliant. If you create a customization on a Liferay JSP but exclude something that provides compliance, you won't be compliant. If you build an ADT or a web content template but do not include necessary attributes, you won't be compliant. If you create custom portlets but do not code per the accessibility guideline, you won't be compliant.

So while Liferay is, itself, compliant to most guidelines, they cannot say the product is compliant because you will ultimately control compliance based upon the content you write and the customizations you implement.

What Build Tool Should We Use?

Liferay still supports the old SDK (although you have to manually download it from github, IIRC) as well as Maven and Gradle. Like every other option, they don't want to pick one and shut out the other developer crowds.

My recommendation is to go with Gradle. It seems to be the trendy option right now and I think its plugins are getting more love these days than Maven. Maven will still work, mind you, but if trends continue to migrate away from Maven (or Gradle for that matter), support for it could go as well.

What Framework Should We Use to Develop Portlets?

There are a myriad of choices here. Classic ones like Spring Portlet MVC and JSF are still supported. Classic Liferay MVC and the new refreshed OSGi flavor is available too.

Liferay has its own JS framework, Metal.js, and a corresponding Soy portlet to leverage there.

Also Liferay has the NPM tools and foundation for building JS-based portlets in Angular, React, Vue, etc.

But no where will they pick one over the others, because again they don't want to shut a crowd out.

I have been known to say, "Code portlets in what you know." The reason here is that if you have a room full of Spring MVC developers, for example, they will become proficient portlet developers faster if you let them use Spring Portlet MVC. If you wanted them to start being Metal.js/Soy portlet developers, they'd have an uphill battle ahead of them.

I've done portlets in many different frameworks over time. At this point, I think the easiest portlets to get right are the Liferay MVC ones based on OSGi. They are extensible, they are typically very simple, they tend to integrate easily in the Liferay environment w/o much third-party dependencies.  With Senna.js, they are also really responsive now and feel on-par with any JS-based portlet framework, but without the complexities of building JS portlets.

The problem, of course, is finding JSP developers. I know there's not many folks out there still doing JSP, so this recommendation does have its challenges.

My backup recommendation though would be Spring Portlet MVC. There's a bigger pool of Spring MVC developers out there, so that solves the resource issue, and Senna.js will still keep the portlets feeling responsive.

Why no cool JS framework? Honestly, it's because of the framework assumptions they carry. For example, they all assume that relative addressing from the current address bar value will work. That might be fine in a standalone JS app, but in the portal it doesn't. The portal address bar is the address for the Liferay page, not your JS assets off of /o/my-js-portlet/... kind of thing. The frameworks just aren't equipped for handling an assigned base address to use in lieu of the address bar value and they honestly have no incentive or desire to support such a thing just for Liferay portlet building.

Where's the Database ERD?

You won't find one in the documentation, it simply isn't there.

Why? Well primarily because Liferay maintains its own data/table relationships internally and they do not conform to normal RDBMS relationships. For example, take the Asset. We all should know that a blog post, a forum post, a web content article, documents and even our own custom entities can be Assets. But that means that in the database, Liferay maintains an abstract relationship between the Asset table(s) and all of these other records. So there is no DB-managed relationship data (no foreign keys, for example) plus all kinds of virtual relationships pretty much connecting everything (same thing goes for permissions, resources, comments, workflow, ...).

Besides, developers typically want the ERD so they can manually manipulate the database. This is always a bad idea because for Liferay, DB persistence is only part of the overall persistence plan. There's index management, cache management, API management (workflow, staging, asset framework, etc), code-based "triggers" via model listeners, etc.  Basically all kinds of things that a direct DB manipulation cannot possibly address, so it is always better to just go through the API.

Additionally, we never ever change the Liferay database DDL either (no adding columns, changing types/sizes, etc.). There are ways to support changes, either via Expandos for existing entities or, my favorite, a custom entity keyed off of the relevant Liferay entity (for easy access). But never a direct DB change.

Conclusion

Well I think that covers the list of unsaid things that I could come up with. I hope that this will clarify some things for everyone.

Think I missed something? Post a comment below and I'll include it in future updates.