How to remove parts of Liferay Portal 7

RE: How to remove parts of Liferay Portal 7

2018-09-17T21:36:32Z

Hi Petr,

I am in somewhat the same boat as you, seeking startup performance improvements.
Especially on the response times of the first request of a portal (an subsequent first page/portlets).
Because we always start fresh build Docker images we 'suffer' from osgi bundles not having state and uncompiled resources.
Which can take up to 2 minutes for a first page to arrive. won't get into details about it, different topic, mybe i write smth about it later.

But regarding " a light-weight Liferay distribution " check this out;
https://dev.liferay.com/develop/tutorials/-/knowledge_base/7-0/liferay-slim-runtime
You maybe could address it from the bottom up. Enabling what you really want instead of

Greeting,
Corné

RE: How to remove parts of Liferay Portal 7

2018-08-24T17:31:00Z

Hi,

I would like to share my own experience with Liferay 7.1.0 and this topic. BTW Out of all the threads ever-returning to this "popular frustration topic", this one seems to be the most complete, so thank you for that. For completeness, here are some of the older threads:

https://community.liferay.com/forums/-/message_boards/message/86270180 "Delete Modules from Liferay 7.0 /osgi/modules Folder Empty"
https://community.liferay.com/forums/-/message_boards/message/76614917 "Exclude Liferay's OSGI bundle on Startup"
https://community.liferay.com/forums/-/message_boards/message/73547796 "Liferay 7 disable features/modules"

Complaints

I think it is a shame that after years, there is still no official (if any) guide on cutting down the Liferay modules, or even better, a light-weight Liferay distribution into which one could simply add the remaining packages. There is even no check for dependent bundles when one removes a bundle (Is it really that hard to implement?!). Hopefully something will emerge in the near future, I guess Liferay authors possibly wait for the community to come with a solution... The bundled Gogo Shell is OK, but not really helpful in many scenarios as well.

Startup times of Liferay 6.2 vs 7.1

Let's be honest, 6.2 started around 1 minute, 7.1 starts around 2 to 3 minutes! And as everyone can see, it's still hard to remove unwanted modules and cut down the startup time. So "reduced startup times ..." is just one big overstatement, but sure, that's needed when one needs to sell a product...

What won't help with startup (but can be seen in some threads)

Increasing -Xmx or -Xms. Default 1G is just enough (tested it).

Setting various "jarsToSkip" properties in Tomcat configuration. Spares maybe one second or two, i. e. nearly useless.

What helps with startup

1) Setting "module.framework.properties.lpkg.index.validator.enabled=false" in Portal properties. Spares around 5s. This is the minimum necessary to always perform some CRC checks. The 2nd function of this setting is skipping a "dependency check" which can take even 4 minutes (Are the authors sure they optimized the checker code enough?!). This check is done only when the bundles change.

2) Removing unwanted packages - spares around 30s if the following LPKGs get removed (didn't try all, but apparently, some need to stay, or stay partly because of dependencies from "core"- didn't try to unpack them to "osgi/modules" and leave just the wanted jars; e. g. removing "Message Boards" results in that the basic "Web Content Display" portlet won't load!):

Liferay CE Collaboration - Liferay CE Adaptive Media - API.lpkg (depends on "Blogs")
Liferay CE Collaboration - Liferay CE Adaptive Media - Impl.lpkg (depends on "Blogs")
Liferay CE Collaboration - Liferay CE Blogs - API.lpkg
Liferay CE Collaboration - Liferay CE Blogs - Impl.lpkg
Liferay CE Collaboration - Liferay CE Knowledge Base - API.lpkg (depends on "Wiki")
Liferay CE Collaboration - Liferay CE Knowledge Base - Impl.lpkg (depends on "Wiki")
Liferay CE Collaboration - Liferay CE Mentions - API.lpkg (depends on "Blogs")
Liferay CE Collaboration - Liferay CE Mentions - Impl.lpkg (depends on "Blogs")
Liferay CE Collaboration - Liferay CE Social - Impl.lpkg
Liferay CE Collaboration - Liferay CE Wiki - API.lpkg
Liferay CE Collaboration - Liferay CE Wiki - Impl.lpkg
Liferay CE Forms and Workflow - Liferay CE Calendar - API.lpkg
Liferay CE Forms and Workflow - Liferay CE Calendar - Impl.lpkg
Liferay CE Forms and Workflow - Liferay CE Polls - API.lpkg
Liferay CE Forms and Workflow - Liferay CE Polls - Impl.lpkg
Liferay CE Forms and Workflow - Liferay CE Portal Reports Engine - API.lpkg
Liferay CE Forms and Workflow - Liferay CE Portal Rules Engine - API.lpkg
Liferay CE Forms and Workflow - Liferay CE Portal Rules Engine - Impl.lpkg
Liferay CE Forms and Workflow - Liferay CE Portal Workflow - API.lpkg
Liferay CE Forms and Workflow - Liferay CE Portal Workflow - Impl.lpkg
Liferay CE Foundation - Liferay CE Screens Compatibility - API.lpkg (depends on "Blogs")
Liferay CE Foundation - Liferay CE Screens Compatibility - Impl.lpkg (depends on "Blogs")

3) Replacing embedded Elasticsearch with the standalone version - spares around 10s. (see the installation steps)

Well, that is probably all for now, looking forward to your feedback... :)

RE: How to remove parts of Liferay Portal 7

2017-09-20T14:10:46Z

David H Nebinger:

It won't do anything to your startup error...

The only thing it does is disable the "security verification" of the lpkg files (a CRC is calculated against the file and this is compared to the local stored value). On my system this was taking a large amount of time and disabling it improved the startup time.

If you look at my original post - that message was generated by the LPKG validation. And it is complaining about missing dependencies, not about the CRC. And as I said - I have tested it yesterday and the startup error is not there with that property set to false.

But you are right - my intention was to improve startup time when turning it off.

RE: How to remove parts of Liferay Portal 7

2017-09-20T13:46:55Z

Tobias Liefke:

Yeah, I did this yesterday, allthough I added it to my system-ext.properties. But this will only suppress the start error. As the dependencies are still missing, the dependent parts still won't work. For example if I set this property and remove "Liferay CE Collaboration.lpkg", I can't even see the page configuration.

It won't do anything to your startup error...

The only thing it does is disable the "security verification" of the lpkg files (a CRC is calculated against the file and this is compared to the local stored value). On my system this was taking a large amount of time and disabling it improved the startup time.

Come meet me at Devcon 2017 or 2017 LSNA!

RE: How to remove parts of Liferay Portal 7

2017-09-20T13:26:22Z

David H Nebinger:

Yeah, that to me was always a red herring. Few of the modules are actually completely independent, there tends to be coupling all over the place. IP Geocoder, for example, is necessary for Audience Targeting. Much is dependent on the collaboration and foundation packages.

I've removed LCS, the connectors (Documentum, Sharepoint and Sync), OAuth provider and the IP Geocoder (because I'm not using AT).

Oh yes, I forgot to say, that I removed these right away. But as these only contain some modules, this didn't do much to startup time. But I hope that in the next version the LPKG packaging is more modular like this and I can for example remove "Liferay CE Wiki.lpkg".

David H Nebinger:

Have you disabled your index validator? Add module.framework.properties.lpkg.index.validator.enabled=false to portal-ext.properties and create an osgi/configs/com.liferay.portal.lpkg.deployer.internal.LPKGIndexValidator.cfg file with the following:
lpkg.index.validator.enabled=false
enabled=false

Yeah, I did this yesterday, allthough I added it to my system-ext.properties. But this will only suppress the start error. As the dependencies are still missing, the dependent parts still won't work. For example if I set this property and remove "Liferay CE Collaboration.lpkg", I can't even see the page configuration.

RE: How to remove parts of Liferay Portal 7

2017-09-19T22:47:19Z

A lot of time ago I heard from dev guys that after splitting everything in OSGI modules, now we are able to modify applications in order to have optional dependences.
For example, if document library modules are not available, web content can work without link to doc. lib. functionality, instead of not starting at all.
But I don't know if it is in the roadmap or it wasn't included.

There is a Liferay Slim project: https://github.com/liferay/liferay-portal/blob/master/SLIM-RUNTIME-README.markdown but it only has the data infrastructure, no UI

RE: How to remove parts of Liferay Portal 7

2017-09-19T16:14:53Z

Tobias Liefke:

Liferay Portal CE 7.0 is also leaner. Its modularized core allows developers and system administrators to remove parts they don’t need or don’t want; this facilitates deployment, reduces startup times and memory footprints, and results in more efficiencies and performance improvements.

Up to now the new architecture has doubled the startup time instead of reducing it. Don't get me wrong - I don't want to complain as every major release of Liferay did give me a reason for getting new development hardware ;-) .

But as I read the mentioned statement I thought about stripping some unused portions from my application, for example Blog, Wiki, Forum, ... nearly everything from Liferay CE Collaboration.lpkg to have a smaller startup time. So I removed that file from $LIFERAY_HOME/osgi/marketplace. Unfortunately there are some reference cycles between that LPKG and Liferay CE Foundation.lpkg, which are printed on startup.

Yeah, that to me was always a red herring. Few of the modules are actually completely independent, there tends to be coupling all over the place. IP Geocoder, for example, is necessary for Audience Targeting. Much is dependent on the collaboration and foundation packages.

I've removed LCS, the connectors (Documentum, Sharepoint and Sync), OAuth provider and the IP Geocoder (because I'm not using AT).

The rest really need to stay.

Have you disabled your index validator? Add module.framework.properties.lpkg.index.validator.enabled=false to portal-ext.properties and create an osgi/configs/com.liferay.portal.lpkg.deployer.internal.LPKGIndexValidator.cfg file with the following:

lpkg.index.validator.enabled=false
enabled=false

My questions now:
Is there a documentation for stripping parts of Liferay (resp. is my strategy correct)?
Is there a dependency graph to find modules which I could remove as well? I know I could find the dependencies with the gogo shell or similar, but I hope that there is already a formatted graph somewhere.
Are there plans to produce LPKGs without dependency cycles in future versions of Liferay?

1. No.
2. No
3. Not that I'm aware of.

Come meet me at Devcon 2017 or 2017 LSNA!

How to remove parts of Liferay Portal 7

2017-09-19T12:21:35Z

On Benefits of Liferay Portal CE 7.0 for Liferay Portal 6 Developers you can find the following statement:

Liferay Portal CE 7.0 is also leaner. Its modularized core allows developers and system administrators to remove parts they don’t need or don’t want; this facilitates deployment, reduces startup times and memory footprints, and results in more efficiencies and performance improvements.

Up to now the new architecture has doubled the startup time instead of reducing it. Don't get me wrong - I don't want to complain as every major release of Liferay did give me a reason for getting new development hardware ;-) .

But as I read the mentioned statement I thought about stripping some unused portions from my application, for example Blog, Wiki, Forum, ... nearly everything from Liferay CE Collaboration.lpkg to have a smaller startup time. So I removed that file from $LIFERAY_HOME/osgi/marketplace. Unfortunately there are some reference cycles between that LPKG and Liferay CE Foundation.lpkg, which are printed on startup, for example:

com.liferay.portal.lpkg.deployer.LPKGVerifyException: LPKG validation failed with {[missing requirement com.liferay.asset.entry.query.processor.custom.user.attributes; version=2.0.4; type=osgi.bundle [caused by: Unable to resolve com.liferay.asset.entry.query.processor.custom.user.attributes version=2.0.4: missing requirement com.liferay.asset.publisher.web.util; version=[1.0.0,2.0.0) [caused by: Unable to resolve com.liferay.asset.publisher.web version=1.5.3: missing requirement &(osgi.extender=jsp.taglib)(uri=http://liferay.com/tld/flags)]]], [missing requirement com.liferay.asset.publisher.web; version=1.5.3; type=osgi.bundle [caused by: Unable to resolve com.liferay.asset.publisher.web version=1.5.3: missing requirement &(osgi.extender=jsp.taglib)(uri=http://liferay.com/tld/flags)]], [missing requirement com.liferay.frontend.editor.alloyeditor.link.browse.web; version=1.0.3; type=osgi.bundle [caused by: Unable to resolve com.liferay.frontend.editor.alloyeditor.link.browse.web version=1.0.3: missing requirement com.liferay.item.selector; version=[1.0.0,2.0.0)]], [missing requirement com.liferay.frontend.image.editor.integration.document.library; version=1.0.11; type=osgi.bundle [caused by: Unable to resolve com.liferay.frontend.image.editor.integration.document.library version=1.0.11: missing requirement com.liferay.document.library.display.context; version=[2.0.0,3.0.0)]], [missing requirement com.liferay.journal.content.web; version=1.1.14; type=osgi.bundle [caused by: Unable to resolve com.liferay.journal.content.web version=1.1.14: missing requirement com.liferay.journal.web.util; version=[1.1.0,2.0.0) [caused by: Unable to resolve com.liferay.journal.web version=1.8.0: missing requirement com.liferay.item.selector; version=[1.0.0,2.0.0)]]], [missing requirement com.liferay.journal.item.selector.api; version=1.0.2; type=osgi.bundle [caused by: Unable to resolve com.liferay.journal.item.selector.api version=1.0.2: missing requirement com.liferay.item.selector; version=[1.0.0,2.0.0)]], [missing requirement com.liferay.journal.item.selector.web; version=1.0.3; type=osgi.bundle [caused by: Unable to resolve com.liferay.journal.item.selector.web version=1.0.3: missing requirement com.liferay.item.selector; version=[1.0.0,2.0.0)]], [missing requirement com.liferay.journal.web; version=1.8.0; type=osgi.bundle [caused by: Unable to resolve com.liferay.journal.web version=1.8.0: missing requirement com.liferay.item.selector; version=[1.0.0,2.0.0)]], [missing requirement com.liferay.layout.item.selector.api; version=1.0.3; type=osgi.bundle [caused by: Unable to resolve com.liferay.layout.item.selector.api version=1.0.3: missing requirement com.liferay.item.selector; version=[1.1.0,2.0.0)]], [missing requirement com.liferay.layout.item.selector.web; version=1.1.5; type=osgi.bundle [caused by: Unable to resolve com.liferay.layout.item.selector.web version=1.1.5: missing requirement com.liferay.item.selector; version=[1.1.0,2.0.0)]], [missing requirement com.liferay.product.navigation.site.administration; version=2.0.7; type=osgi.bundle [caused by: Unable to resolve com.liferay.product.navigation.site.administration version=2.0.7: missing requirement com.liferay.item.selector; version=[1.1.0,2.0.0)]], [missing requirement com.liferay.site.item.selector.api; version=1.0.3; type=osgi.bundle [caused by: Unable to resolve com.liferay.site.item.selector.api version=1.0.3: missing requirement com.liferay.item.selector; version=[1.1.0,2.0.0)]], [missing requirement com.liferay.site.item.selector.web; version=2.0.9; type=osgi.bundle [caused by: Unable to resolve com.liferay.site.item.selector.web version=2.0.9: missing requirement com.liferay.item.selector; version=[1.1.0,2.0.0)]]}

As a result I changed my strategy, extracted the original "Liferay CE Forms and Workflow.lpkg" into osgi/modules and removed only some parts, especially com.liferay.blogs.*, com.liferay.message.boards.* , com.liferay.wiki.*

The good news: The server started and the whole startup wasn't slower than before.
The bad news: It spared me only 6 seconds which is nearly at the measuring threshold for a startup time of 230 seconds (only measured once).

My questions now:

Is there a documentation for stripping parts of Liferay (resp. is my strategy correct)?
Is there a dependency graph to find modules which I could remove as well? I know I could find the dependencies with the gogo shell or similar, but I hope that there is already a formatted graph somewhere.
Are there plans to produce LPKGs without dependency cycles in future versions of Liferay?