Pre-Upgrade Scripting (6.2 > DXP) - Pt 1 - Delete Layouts

Getting ready to upgrade to DXP from Liferay 6.2? You reached the section of the documentation about pre-upgrade preparations that will speed up the process? Me, too. With much appreciation to Sébastien Le Marchand for his excellent article "5 tips to improve usage of the Liferay script console" and Corné Aussems for his public groovy scripts that propelled me along.

I would like to offer some additional scripts that do achieve some of the tasks recommended in Liferay's pre-upgrade tips as they affect our particular environment. We will use the Server Administration > Script Console for our work. And, we will always use Sebastian's "preview mode" feature as we test run our scripts.

This blog entry will be about bulk-removing pages (LAYOUT) and targeting a collection of pages using ActionableDynamicQuery API. We have surpassed 80,000 users and each user is issued a personal site of 3 pages with several portlets deployed on each. It turns out our audiences are not using the platform for this feature and we will look to remove most, if not all, the pages of each of the personal sites. This will, in turn, help to reduce our resourcepermission table's 2 million+ entries. We are fairly certain we will at least delete 2/3 pages - each named "Blog" (/blog) and "Friends," (/friends) respectively.

Let's see what we have for some counts before we begin:

mysql> select count(*) from layout where friendlyURL = '/blog'; (Returns: 81350)
mysql> select count(*) from layout where friendlyURL = '/friends'; (Returns 81354)
mysql> select count(*) from resourcepermission; (Returns 2,115,503)

Wow - that's a lot of layouts! We'll want to make sure the script runs through the entire database before crapping out.

We can thank Michael Bowerman for contributing significant improvements to the efficiency and flexibility of an earlier version. Let's take a look over the final script and Michael's great advices regarding Liferay's "DynamicQuery" along with the "Disjunction" functionality to delete all pages of either friendlyURL...:

By using ActionableDynamicQuery, you can fetch and process objects in smaller batches, substantially improving performance. To use the ActionableDynamicQuery method, you would first create a new ActionableDynamicQuery object corresponding to the type of model you want to process. In this case, the type of model you are processing is Layout, so you would create a LayoutActionableDynamicQuery object (in the com.liferay.portal.service.persistence package). When you create the ActionableDynamicQuery object, you can override the default methods from the BaseActionableDynamicQuery class. The two methods you'll probably be interested in overriding are:

1. public void addCriteria(DynamicQuery dynamicQuery)
- What the addCriteria method does is it allows you to filter down criteria for objects when fetching them. If you do not implement this method, then all objects of that model type in the database will be fetched. If you do implement the method, you can use it to specify criteria by which objects will be fetched (for instance, in this case, we would want to use this method to only fetch Layouts whose friendlyURL is "/friends"). You will be using hibernate's criteria API when you implement this method
AND
2. protected void performAction(Object object)
- The performAction method specifies the action to be performed on each object that is fetched. In this case, the object we have fetched is a layout, and the action we will want to perform on it is to delete that layout.

Once you have created the ActionableDynamicQuery object that implements these two methods (or at least implements the performAction method), then you can simply invoke actionableDynamicQuery.performActions to fetch the entries and invoke the action on them.

The advantage of ActionableDynamicQuery is that it only fetches in batches, using a pre-set size (by default, 10000). Once it finishes processing the first batch, it then fetches the second batch to be processed, and so on and so forth, until it has processed every batch. In comparison to fetching every object into a single list, the performance improvement can actually be quite large depending on how many objects you are fetching.

For further exploration of DynamicQuery API in Liferay:

The script took some time to run so, we split the work into two scripts (1 for "/blog' pages and one for '/friends' pages). An exception occurred and when I dug in I noticed the failed layout's page owner had changed the page type of one of the pages to URL page type and thus became undeleteable. Other than that, these scripts wiped out all the desired layouts and reduced the resourcepermission table by more than 50%. Win.

ResourcePermission Table Before: 2,115,503. After: 898,188!

Cheers. I'll share my next decent script after I complete it...

2 Comments

Please sign in to comment.

Hi Daniel,

Thanks for sharing your knowledge about how to do some pre-upgrade cleaning to make the upgrade process go more smoothly! One way to improve your script might be to improve the memory usage of it. I noticed you had this line in your script:

layouts = LayoutLocalServiceUtil.getLayouts(-1,-1)

As you know, this API will fetch every layout in the database into one giant list. Since this has the potential to consume a very large amount of memory (for instance, in your example, the list would be over 2 million entries long), this could be a hindrance to performance since there is less available space for everything else and the garbage collector will need to be run more frequently. It also takes a long time just to generate that list.

Liferay actually has an API called ActionableDynamicQuery which you can use for situations like these. By using ActionableDynamicQuery, you can fetch and process objects in smaller batches, substantially improving performance. To use the ActionableDynamicQuery method, you would first create a new ActionableDynamicQuery object corresponding to the type of model you want to process. In this case, the type of model you are processing is Layout, so you would create a LayoutActionableDynamicQuery object (in the com.liferay.portal.service.persistence package). When you create the ActionableDynamicQuery object, you can override the default methods from the BaseActionableDynamicQuery class. The two methods you'll probably be interested in overriding are:

1. public void addCriteria(DynamicQuery dynamicQuery)
- What the addCriteria method does is it allows you to filter down criteria for objects when fetching them. If you do not implement this method, then all objects of that model type in the database will be fetched. If you do implement the method, you can use it to specify criteria by which objects will be fetched (for instance, in this case, we would want to use this method to only fetch Layouts whose friendlyURL is "/friends"). You will be using hibernate's criteria API when you implement this method
AND
2. protected void performAction(Object object)
- The performAction method specifies the action to be performed on each object that is fetched. In this case, the object we have fetched is a layout, and the action we will want to perform on it is to delete that layout.

Once you have created the ActionableDynamicQuery object that implements these two methods (or at least implements the performAction method), then you can simply invoke actionableDynamicQuery.performActions to fetch the entries and invoke the action on them.

The advantage of ActionableDynamicQuery is that it only fetches in batches, using a pre-set size (by default, 10000). Once it finishes processing the first batch, it then fetches the second batch to be processed, and so on and so forth, until it has processed every batch. In comparison to fetching every object into a single list, the performance improvement can actually be quite large depending on how many objects you are fetching.

I've created an example script based on your script: https://gist.github.com/mbowerman/709286728084081ade7d2afbc72f1bb1. Please feel free to take a look at it ; hopefully it's clear how it all works.

You can do a lot more complicated things with the addCriteria method than the example in that script. If you call dynamicQuery.add on other criteria, it functions as an AND (so each condition must be true for the object to be fetched). You can utilize ORs by using the Disjunction object. You can utilize other condition types besides equal (for instance, less than, greater than, not equal to, or is a member of a list). You can even add subqueries! It's a bit difficult for me to explain in words how to do all these things, but please let me know if you'd like an example of any of them.

I also found this documentation which has more information on using ActionableDynamicQuery (although it is for 7.0, so some of the information may not apply the same to 6.2): https://dev.liferay.com/es/develop/tutorials/-/knowledge_base/7-0/dynamic-query

Please sign in to reply.

Thank you very much, Michael! I had been wanting to explore DynamicQuery and your guidance and example are very valuable tips for improving this and many other scripts. The time you took to do this means a lot to me and I am most grateful. You certainly made my day. -daniel

Please sign in to reply.

Menu Display

Blogs

Category Filter

More Blog Entries

New Installers and IDE 3.2.0 Milestone 1 Released

Pre-Upgrade Scripting (6.2 > DXP) - Pt 2 - Document Version Cleanup