Blogs

Blogs

New Since Liferay 7.1 - Localized Entities

Localized Entities are a new way to support localization on your ServiceBuilder entities

Classic Liferay Localization Handling

Any developer who has been around for awhile will have some knowledge of how Liferay handled localization for the entities...

In your service.xml file where you normally just have a column type of String, you would also have an additional attribute set, localized="true".

With this addition, your model classes getters and setters change from using a simple String instead to a Map<String,String> for localized versions. The key for the map is the language ID (i.e. "en_US") and the value is the localized version for that language.

The database storage has always been the challenging part. The column in the database is a CLOB type, and the value is a big chunk of XML that represents the Map data.

This kind of storage poses a couple of problems:

  • CLOBs are not unlimited in size. A fairly large chunk of content translated into a fairly large set of languages can, in some cases, blow out CLOB sizes in some databases.
  • Simple SQL where clause searches are not possible. For example, try coming up with a query to find all records that have not yet been translated into Italian? It sounds kind of easy, but you should remember that not all of your SQL-92 directives are going to work on a CLOB directly, so it may not be as easy as you think.
  • Accessing content for a single language always comes with overhead to retrieve the record, marshal the XML back into the Map and then extract the content for the given language ID.
  • An entity with four localized columns has these issues multiplied by four, so while the impact is arithmetic, that cannot be overlooked from a performance perspective.

While the problems are known, it was still the only game in town for supporting localization of content.

Until, that is, Localized Entities were added to Liferay 7.1.

Localized Entities to the Rescue

Localized Entities are a new column type added for the <entity /> tag in Liferay 7.1. Much to my surprise, these things are not documented anywhere. In fact, I found it totally by accident while I was looking at the User-Associated Data (UAD, those services added for GDPR compliance) attributes in the ServiceBuilder DTD.

Instead of defining your column as:

<service-builder dependency-injector="ds" package-path="foo">
	<namespace>FOO</namespace>
	<entity local-service="true" name="Foo" remote-service="true" uuid="true">

		<column name="fooId" primary="true" type="long" />

		<!-- Group instance -->

		<column name="groupId" type="long" />

		<!-- Audit fields -->

		<column name="companyId" type="long" />
		<column name="userId" type="long" />
		<column name="userName" type="String" />
		<column name="createDate" type="Date" />
		<column name="modifiedDate" type="Date" />

		<!-- Other fields -->

		<column name="title" type="String" localized="true" />
		<column name="synopsis" type="String" localized="true" />
		<column name="description" type="String" localized="true" />
		<column name="tease" type="String" localized="true" />
		
		<order>
		...

You'd lay it out as:

<service-builder dependency-injector="ds" package-path="foo">
	<namespace>FOO</namespace>
	<entity local-service="true" name="Foo" remote-service="true" uuid="true">

		<column name="fooId" primary="true" type="long" />

		<!-- Group instance -->

		<column name="groupId" type="long" />

		<!-- Audit fields -->

		<column name="companyId" type="long" />
		<column name="userId" type="long" />
		<column name="userName" type="String" />
		<column name="createDate" type="Date" />
		<column name="modifiedDate" type="Date" />

		<!-- Other fields -->

		...

		<!-- Localized fields -->

		<localized-entity>
			<localized-column name="title" />
			<localized-column name="synopsis" />
			<localized-column name="description" />
			<localized-column name="tease" />
		</localized-entity>
		
		<order>
		  ...

The localized entities become, well, entities of their own, complete with their own persistence class. In your model, you'll have Foo and FooLocalization. You'll get a FooPersistence and a FooLocalizationPersistence. You'll also have a FooLocalServiceImpl, but you won't find a FooLocalizationLocalServiceImpl though. It's up to you to expose access to the localized values through your FooLocalServiceImpl.

Your Foo model will still have the same kind of getters and setters that you're used to seeing from the old localization pattern, but the implementation instead is using the FooLocalServiceUtil to invoke custom methods on FooLocalService which in turn invoke the FooLocalizationPersistence to fetch and store data.

In the database, you'll find that you have a new table. Instead of just the FOO_Foo table, you'll also have FOO_FooLocalization. This table has a column for the language id the row represents and a CLOB for the value(s). In the example above, the entity would have four CLOBs, one for each field. The FooLocalization has a key for the primary key for Foo (a one-to-many mapping allowing each Foo to have multiple FooLocalization records). Each field will be its own CLOB for storing just the content, not a Map w/ all languages.

Both Foo and FooLocalization entities will have some additional fields not specified in service.xml that are used by the implementations; some of these you will need to assign values for, some will just be handled for you.

With the addition of Localized Entities, our original storage problems have all been addressed:

  • CLOBs are still used, but the limitation is now only the size of the content itself, not whether the Map XML must be stored there.
  • As the table has a separate row per language id, SQL queries that were impossible (or just impractical) before now become possible and, in some cases, downright easy.
  • Accessing content for a single language is now a query against FooLocalizationPersistence for the right Foo primary key and the language ID that is necessary (a method that FooLocalizationPersistence already has implemented for you) to retrieve a FooLocalization and return the necessary field.
  • Since all localized fields for one language ID is in one table, accessing all of the values is just a matter of accessing the FooLocalization entity, so regardless how many fields there are the performance characteristics are simply flat.

Additionally, we have some new capabilities that we might want to consider:

The <localized-entity /> tag can have non-localized <column /> tags. Imagine if you wanted to track the number of views per language ID (so you could see which language was getting the most views). We could modify the entity like:

	<localized-entity>
		<column name="views" type="long" />
		<localized-column name="title" />
		<localized-column name="synopsis" />
		<localized-column name="description" />
		<localized-column name="tease" />
	</localized-entity>

This new column will be part of our FooLocalization entity, and since we can access it and persist it, we can start tracking localized views per language id.

There's many other use cases for storing additional data with a language id, the limit is just going to be your own imagination.

Localized entities can also have their own <order /> and <finder /> stanzas, so you can add as necessary to get the kind of retrieval options you would use for a regular entity.

Some Assembly Required

So you may be thinking "Wow, these new Localized Entities are great, all of my localization problems will be solved!"

Unfortunately, they are not new magical tags that can do everything for you.

Persistence, in fact, is still mostly manual. In my implementations, I still build out my addFoo(...) method, but now following Liferay's lead I'll do something like:

public Foo addFoo(..., final Map<String,String> titleMap, final Map<String,String> synopsisMap, final Map<String,String> descriptionMap, final Map<String,String> teaseMap, ...) {
    String defaultLanguageId = LocaleUtil.toLanguageId(LocaleUtil.getSiteDefault());
    
    return fooLocalService.addFoo(..., defaultLanguageId, titleMap, synopsisMap, descriptionMap, teaseMap, ...);
}

public Foo addFoo(..., final String defaultLanguageId, final Map<String,String> titleMap, final Map<String,String> synopsisMap, final Map<String,String> descriptionMap, final Map<String,String> teaseMap, ...) {
    ...
}

This follows Liferay's basic implementation for FriendlyURLEntry (the only Liferay usage example I could find) by allowing the default language ID to be defined externally.

There's no built-in persistence support for the localization maps, so I also have to persist these manually:

protected void updateLocalizations(final Foo foo, final Map<String, String> titleMap, final Map<String, String> synopsisMap,
								 final Map<String, String> descriptionMap, final Map<String, String> teaseMap, final ServiceContext serviceContext) {

	int changes = 0;

	// we need a set for the given languages. Could use all supported languages of the company and/or platform, but why
	// create rows we don't need.
	Set<String> languageIds = new HashSet<>();

	if (titleMap != null) {
		languageIds.addAll(titleMap.keySet());
	}
	if (synopsisMap != null) {
		languageIds.addAll(synopsisMap.keySet());
	}
	if (detailMap != null) {
		languageIds.addAll(descriptionMap.keySet());
	}
	if (detailMap != null) {
		languageIds.addAll(teaseMap.keySet());
	}

	FooLocalization fooLocalization;
	boolean changed = true;

	for (String languageId : languageIds) {
		fooLocalization = fooLocalizationPersistence.fetchByFooId_LanguageId(foo.getFooId(), languageId);
		changed = false;

		if (fooLocalization == null) {
			fooLocalization = fooLocalizationPersistence.create(counterLocalService.increment(FooLocalization.class.getName()));
			changed = true;

			fooLocalization.setFooId(foo.getFooId());
			fooLocalization.setLanguageId(languageId);

			fooLocalization.setCompanyId(serviceContext.getCompanyId());
			fooLocalization.setViews(0);
		}

		if (isChanged(localeId, fooLocalization.getTitle())) {
			changed = true;
			fooLocalization.setTitle(titleMap.get(localeId));
		}
		if (isChanged(synopsisMap.get(localeId), fooLocalization.getSynopsis())) {
			changed = true;
			fooLocalization.setSynopsis(synopsisMap.get(localeId));
		}
		if (isChanged(descriptionMap.get(localeId), fooLocalization.getDescription())) {
			changed = true;
			fooLocalization.setDescription(descriptionMap.get(localeId));
		}
		if (isChanged(teaseMap.get(localeId), fooLocalization.getTease())) {
			changed = true;
			fooLocalization.setTease(teaseMap.get(localeId));
		}

		if (changed) {
			changes++;
			challengeLocalizationPersistence.update(fooLocalization, serviceContext);
		}
	}

	if (_log.isDebugEnabled()) {
		_log.debug("There were {} changes made to foo localizations.", changes);
	}
}

Seems kind of long, but I can invoke this method at the end of either an addFoo() or updateFoo() method and the localizations will be updated as necessary.

Also, I was kind of shocked to find that not even deletions are handled for you. So you need to override the various deleteFoo() methods and add the following:

fooLocalizationPersistence.removeByFooId(foo.getFooId());

Because I prefer to have all access to inner classes (finders, custom sql, and now the localization persistence) hidden behind my FooLocalService, I also add utility methods like:

public FooLocalization fetchFooLocalization(final long fooId, final String languageId) {
	FooLocalization localization = fooLocalizationPersistence.fetchByFooId_LanguageId(fooId, languageId);

	if (localization == null) {
		// if not found, try the default language.
		Foo foo = fetchFoo(fooId);

		if (foo != null) {
			localization = fooLocalizationPersistence.fetchByFooId_LanguageId(fooId, foo.getDefaultLanguageId());
		}
	}

	return localization;
}

/**
 * updateFooLocalizationViews: Fetches the view for the given foo and language id. Also will update the
 * view count, but will not count a default language view as a view since it will artificially skew the
 * view numbers.
 */
public FooLocalization updateFooLocalizationViews(final long fooId, final String languageId) {
	FooLocalization localization = fooLocalizationPersistence.fetchByFooId_LanguageId(fooId, languageId);

	if (localization == null) {
		// if not found, try the default language.
		Foo foo = fetchFoo(fooId);

		if (foo != null) {
			localization = fooLocalizationPersistence.fetchByFooId_LanguageId(fooId, foo.getDefaultLanguageId());
		}
	} else {
		// we don't want to count a default view because it will skew the numbers
		localization.setViews(localization.getViews() + 1);
		
		localization = fooLocalizationPersistence.update(localization);
  }
  
	return localization;
}

With these kinds of methods in place, I find it easier to update the FooImpl model class to invoke FooLocalServiceUtil to extract the content I need when I need it, and it also encapsulates my "business logic" to hide it from outside callers.

Conclusion

So this is the new Liferay way for handling localization in your content. It solves some of the problems with the old way of handling localizations while at the same time opening doors for some new and exciting possibilities.

And, since the code mostly conforms to how normal models and services are built, the regular developer extension points are still available. Heck, you can add ModelListener<FooLocalization>'s if you really want to.

So while there is some effort on your part to add in support for the new localization persistence mechanism, I think at the end of the day following this pattern will lead you to much better outcomes vs the old way.

Blogs

Hi David,

 

Thanks for your blog post, this helped us a lot!

 

You wrote: "There's no built-in persistence support for the localization maps, so I also have to persist these manually"

 

Maybe this has changed since the latest service builder, but we got a "updateMyEntityLocalizations(MyEntity, Map<String, String> ...)" method generated by the service builder. So no need for us to implement this manually.

 

The only problem that we had is that the maps we got from the frontend are of type Map<Locale, String>, like:

 

    Map<Locale, String> titleMap = LocalizationUtil.getLocalizationMap(actionRequest, "titleMapAsXML");  

So we had to convert this map into Map<String, String> to use the generated update localizations method via:

 

    private Map<String, String> toStringMap(Map<Locale, String> localeMap) {                  Map<String, String> stringMap = new HashMap<>();                  localeMap.forEach((locale, string) -> {             if (string != null && !"".equals(string)) {                 stringMap.put(locale.toString(), string);             }         });                  return stringMap;     }

 

Is there a reason these two map types differ? Has the method changed to get the localization map from the request? Do we miss something here?

 

Greetings,

Mirko

Hi David

Thanks for the great article.

About challenges, you have mentioned for setting localized="true" for localized fields, if there are use cases that do not have those challenges like, small blobs, or not having complex queries, can we still use  localized="true" for localized fields or just we have to use new feature <localized-entity> and Liferay might deprecate localized="true" in future?because using <localized-entity> also has challenges of adding a new localized table in the database for each entity and adding about 12 extra java files for each <localized-entity> and making CRUD services more complex and manual.